mirror of
https://github.com/Stirling-Tools/Stirling-PDF.git
synced 2026-09-03 05:10:16 +03:00
update_python_dep_20260812
275
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
79686a3a09 |
form field editing (#6655)
# Description of Changes Building ontop of a users draft PR for form creation tools **Fill Form** becomes a full **Form Editor**: fill, create, modify and delete AcroForm fields visually. Builds on the community form-creation draft, plus a UX/UI rework pass. - **Backend**: `/api/v1/form` endpoints — `fields-with-coordinates`, `add/modify/delete-fields`, combined `edit-fields` (one round-trip), `fill`, `extract-csv/xlsx`; supports text (multiline, comb), checkbox, dropdown, list box, radio, button actions (reset/print/URL/submit) and signature placeholders - **Create**: type palette, click-or-drag placement with snap guides, inline property editor, batch "Add N fields" - **Modify**: move/resize on the page, arrow-nudge + Delete key, X/Y/W/H inputs, staged edits/deletes with chips, discard - **Fill**: live progress + required tracking, flatten toggle, Export menu (JSON/CSV/XLSX), Ctrl/Cmd+S - **Safety**: confirm dialog before discarding staged work; empty required fields warn with "Save anyway" instead of blocking - **UI**: consistent panel skeleton (fixed header / scrolling list / pinned actions), empty states that link into Create, full i18n with plural keys [walkthrough.html](https://github.com/user-attachments/files/30508976/walkthrough.html) --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. --------- Co-authored-by: Denys Vitali <denys@denv.it> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
James Brunton
|
158187ac46 |
Fix Documents tab in Processor (#7569)
# Description of Changes The Documents tab in the Processor is supposed to be available to all Processor users, but because the API is built on top of the Audit data, which is only for enterprise users, the API call always fails with 403. This means that it never fills the query cache, so every time you go back to the tab it has to reload all the data for a couple of seconds (and will fail again). This fixes the API so that it's available to any Processor user instead of just enterprise users. Also, the documents data was only being written to the log on an enterprise license, so I've changed it so that data is always tracked in the audit log because otherwise the Documents tab would still be useless to non-enterprise users. The Audit Log tab was also available to all Processor users, but would have the same issue where the table would never load because the API would 403 as well. I've just made the Audit Log tab disabled for non-enterprise users now. We might want to do something to signpost it a bit more that it's an enterprise-specific feature, but it's better than nothing for now. |
||
|
Anthony StirlingandJames Brunton
|
e50c3de0a9 |
Run classification locally first and only escalate an unsure verdict to the AI (#7580)
Split out of #7574 — this is the classification half, which is independent of the editor-source work and can land on its own. ## What this does - **Runs the local heuristic first and only escalates an unsure verdict to the AI.** A high-confidence local answer stands; anything less (or a file the heuristic hasn't reached yet) goes to the engine. A wrong label costs more than an engine call, so the bar is deliberately strict. - **Makes `classify` an authorable pipeline task**, so it can be used as a step like any other tool, and skips files that are already classified. - **Leaves the seeded Classification policy unowned** rather than naming a `system` placeholder that was never a real user; existing seeds are repaired on boot. ## Review feedback applied From @jbrunton96 on #7574: - **The generic runner no longer names classification.** Everything classification-specific moved into `proprietary/data/classificationPolicy.ts`, and `usePolicyAutoRun` now asks capability questions instead: `policyRewritesDocument`, `policyDeliversOutputFiles`, `policyRequiresAiEngine`, `shouldDispatchToAi`. There is no `id === "classification"` left in the runner. - **Ordering is no longer a name in the runner.** `pinClassificationLast` is gone; the runner sorts annotating policies after rewriting ones. The constraint is real: an annotating policy is non-blocking, so a rewriting one running after it forks from the pre-annotation version and drops the labels. To be straight about what this is and isn't - see "Still open" below - `policyRewritesDocument` is still keyed on the category id, not on a property each policy declares. The check moved out of the runner; it did not stop being a check on one id. - **Confidence is typed.** New `ClassificationConfidence` union in `core/types/fileContext.ts`, reused by `fileStorage`, `HeuristicConfidence`, and the trusted-verdict constant instead of being respelled at each site. - **Comments trimmed** to the repo's 2-line guideline, and a stale seeder javadoc that still claimed an internal-user owner was corrected. ## Still open, deliberately `classificationPolicy.ts` answers its capability questions with `categoryId === "classification"`. That is the same check relocated, not removed, and the module doc now says so outright. Deliberate, for two reasons: - **The concept it would be declared against is going away.** Policies are becoming pipelines with labels behind a separate enforcement layer, which removes the category the flag would live on. A capability system built on `categoryId` today gets migrated twice. - **Classification is genuinely privileged, not accidentally special.** It is the only policy with a browser-side implementation, so it can answer without the server. That is a product decision, and a local-only mode for set scenarios is planned - the flag for it should be designed with that feature, not guessed at now. The end state for the rest: an in-place output mode retires the ordering rule and `policyDeliversOutputFiles`, and a run result that can carry findings as well as files retires the remainder. Both touch the import path, which is the most delicate code in `usePolicyAutoRun` - not something to bolt on to a PR that has already been split once. Nothing is broken by leaving it. A user-built classify pipeline still gets its labels: the generic import path reads them off the returned PDF. It versions the file instead of labelling in place, and it misses the local-heuristic shortcut, so it always bills the engine. ## Testing - `classificationPolicy.test.ts` — 12 cases covering each capability and the escalation rule - Full frontend `proprietary` project: 39 files / 442 tests - `:proprietary:test` for `DefaultClassificationPolicySeederTest` + `ClassifyLabelControllerTest` - `tsc --noEmit` on core, proprietary, portal, saas, desktop, cloud --------- Co-authored-by: James Brunton <jbrunton96@gmail.com> |
||
|
Anthony Stirling
|
5f9c396fdd | test: remove PdfUaBenchmarkTest (#7613) | ||
|
dependabot[bot]
|
a7c6fa6ef6 |
build(deps): bump io.swagger.core.v3:swagger-core-jakarta from 2.2.46 to 2.2.53 (#7526)
Bumps io.swagger.core.v3:swagger-core-jakarta from 2.2.46 to 2.2.53. [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
Anthony Stirling
|
96a00cebd1 |
Pdf ua converter testing (#7301)
# Description of Changes Adds a PDF/UA converter, an accessibility report, and PDF/A conformance level A. **New: `POST /api/v1/convert/pdf/ua`** (Convert tool, "PDF/UA" target). Tags an untagged PDF, marks decorative content as artifacts, embeds missing fonts and applies the document-level PDF/UA requirements (title, language, tab order, form-field descriptions), then validates with veraPDF. The `pdfuaid` declaration is written only if validation passes, so a returned file never claims more than it delivers; response headers report whether it was declared, how many checks still fail and how many images still need a description. **New: `POST /api/v1/security/accessibility-report`.** Reports what fails, what the converter can fix on its own, what needs a person, and lists the figures needing a description with the keys the conversion accepts back. Read-only; does not modify the file. Capped at 100 MB / 2000 pages and weighted `LARGE_WEIGHT`, since it runs a full veraPDF pass plus the converter's layout analysis over every page. **PDF/A level A.** `pdfa-1a`, `pdfa-2a` and `pdfa-3a` output formats on the existing `/api/v1/convert/pdf/pdfa` endpoint. Level A is level B plus tagging, so the document is tagged after Ghostscript (which discards any structure tree it is given) and the level A claim is written only if veraPDF agrees. Optional `pdfUa=true` additionally declares PDF/UA alongside PDF/A, again only if it validates. Honesty rules the implementation holds to: - **Never claim a level that was not reached.** If tagging fails, the file is returned at level B and is named `_PDFA-2b.pdf`, not `_PDFA-2a.pdf`. With `strict=true` the request fails outright rather than returning a level B file against a level A request, and a level B pass no longer satisfies a strict level A request. - **Never relabel a document's language.** The requested language (default `en-GB`) is applied only when the document declares none; a French PDF stays French unless the caller sets `overrideLanguage`, and ignoring a requested language is reported as a warning. - **Never invent alternative text.** Descriptions come from the caller. The Convert panel can list the images needing one (via the report endpoint) and send them back per figure; any image left undescribed blocks the conformance claim rather than being papered over. - **Never certify hidden content.** Marking images decorative, or suppressing text that could not be tagged reliably, withdraws the claim instead of passing the checker by hiding content. PDF/UA-1 and PDF/UA-2 are both offered; UA-2 raises the file to PDF 2.0 and namespaces the structure tree, and its test asserts conformance rather than merely reporting it. Convert steps saved in Automations/Pipelines round-trip their PDF/UA settings (profile, language, override, title, font embedding, descriptions). --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
James Brunton
|
a744102cb6 |
Support Supporting Files in Pipelines (#7547)
# Description of Changes Currently in the Processor's Pipelines page, none of the tools which require supporting files are usable because it's never been hooked up to the new API to upload supporting files. This PR hooks it up to that so all tools using supporting files work in the processor. I had to tweak the type generation a little for this so we have a static map of which params are for supporting files so we know to handle them differently. The `Test with a file` button has to work a little differently than the main run since it's running an ad-hoc pipeline so the files haven't necessarily been saved to the server yet. In this case, it'll use whatever local changes the user has made for those pipeline steps, and for all other steps, it'll just use what's saved in the server. |
||
|
EthanHealy01
|
0f8803f35f |
Require the policy-management role to run a policy against its sources (#7565)
## What
Running a stored policy against its **configured sources** (`POST
/api/v1/policies/{id}/trigger`, the manual "run now") now requires the
policy-management role — global admin self-hosted, team leader on SaaS —
alongside the existing team scoping.
## Why
A source sweep operates on the team's configured sources using the
server's stored connection credentials, so it belongs with the other
policy-management capabilities rather than with ordinary use. Team
scoping on its own didn't express that distinction.
## Not changed
- `POST /{id}/run` — running a policy over documents the **caller
supplied** stays open to every team member. That's ordinary editor
enforcement on upload and export, and gating it would break it.
- Ad-hoc pipelines (`/run`, `/run/stream`).
- The scheduled, folder-watch and webhook triggers.
- Single-user deployments (login disabled), which have no roles.
## Implementation
`PolicyManagementAuthority` gains `canTriggerPolicies()`, kept separate
from `canEditPolicies()` so the two capabilities can diverge later. Both
current implementations grant it to the same principals that may edit
policies.
## Tests
- role absent → 403, rejected before any run starts
- role present → 202
- login disabled → check skipped entirely
- `/{id}/run` asserted to consult neither authority method, so the gate
can't quietly extend to the editor path later
|
||
|
EthanHealy01
|
08b08aa8a1 |
Let everyone read the failures they caused (Review Flow PR 3) (#7477)
Review Flow PR 3. Stacked on #7296. A recorded failure becomes readable by the person who caused it. ## What changes Before this, reading or triaging a failure required leader permissions: `FileRunEventController.requireFailureReviewAllowed()` returned 403 to anyone who could not edit policies. #7296 lets any user report a failure, so they could file into a queue they could never read. That gate is removed from the endpoints and the decision moves into `FileRunEventService`: | Caller | Reads and closes | |---|---| | Team leader or admin | the whole team's failures (unchanged) | | Anyone else | only failures where `actor` is them | | Team unresolvable | nothing | | Name unresolvable | nothing | `GET /kinds` is also opened. It returns static enum metadata, and a member needs it to render failures they can already see. ## Additions - An `actor` predicate on both list queries in `FileRunEventRepository`, threaded through `FileRunEventStore.list`. - `ReadScope` (permitted, teamId, actor) replacing `TeamScope`, with `wholeTeam` / `mine` / `denied` factories. - An actor filter on `dispatch`, so acting on another person's row answers **404, not 403** — the same response as an id that does not exist. ## Fixes - **`report()` filed rows under the wrong team.** It took the team from the read scope, which returns null for a caller who cannot be named, so such a report landed unteamed in the bucket every team shares. It now uses a dedicated `currentTeamId()`. - **`forgetFiles` narrows to the caller even for a leader.** File ids are minted by each client, so scoping on team alone would let one caller close a colleague's incidents by naming ids. - The controller no longer injects `PolicyManagementAuthority` or `ApplicationProperties`; with the gate gone it decides nothing. ## Team isolation Unchanged and covered by database-backed tests rather than mocks. `FileRunEventStoreDbTest` asserts that a caller with a team sees only their own team's rows and never the unteamed ones, and that the actor predicate narrows within a team without ever widening across one. Delete either clause from the JPQL and one of those tests fails. No endpoint accepts a team parameter; the team always comes from the authenticated principal. **Attribution is fixed here too, because this PR depends on it.** A failure's actor was read from the MDC audit principal, which carries the BILLING identity — for a stored policy, always its owner. Since reads are now narrowed to the rows you are the actor on, a wrong actor means the member who caused a failure and holds the document reads nothing, while the policy owner is handed incidents from runs they never triggered. The triggering user is now carried on the run, separate from the billing principal and the output owner, and is null for a trigger-fired sweep so an unattended failure stays ownerless. `PolicyFailureAttributionTest` runs the real engine, recorder, store and service together. The two sides used to assert independently — the engine's test matched the actor with `any()`, which is how this went unnoticed. ## How to test Needs a proprietary or SaaS build with login enabled and two accounts in the same team, one a leader and one not. `task dev:all` gives you the stack. 1. **As the member**, fail a tool: open a PDF and run **Remove Password** with a wrong password. 2. **Still as the member**, go to `/processor/documents` → **Failures**. Before this PR you got nothing here. Now you see your own row, and only yours. 3. **As the leader**, open the same view. You see the whole team's rows, including the member's. 4. **Member cannot reach a colleague's row.** As the leader, copy a row's id from **Show raw JSON**. As the member, `POST /api/v1/file-run-events/{thatId}/actions/DISMISS`. It answers **404**, and the row is untouched — it must not answer 403, which would confirm the row exists. 5. **Member can close their own.** Dismiss your own row as the member. It leaves the default view. 6. **Deleting a file only closes your own rows.** As the leader, delete a file in your editor. The member's incidents are untouched even if the leader's client happened to name the same ids. ## Migration None. `actor` is an existing column; this only adds predicates to existing queries. |
||
|
EthanHealy01
|
2483e9f37a |
Report editor-originated failures into the same queue (Review Flow PR 2) (#7296)
Review Flow PR 2 of 5. Editor tool failures now reach the same durable queue as failures from folders, buckets and webhooks. ## What's added **A report endpoint** — `POST /api/v1/file-run-events/reports`, open to any authenticated user. Takes four fields: `operation`, `errorCode`, `fileIds`, `detail`. No team, no actor, no filename: the first two come from the session, the third is never a field. Refused with 400 above 200 file ids, and nothing is written when refused. **Automatic reporting from every tool** — wired into `useToolOperation`, so no per-tool work is needed. Client-side refusals (an unsupported format that never reaches the server) are reported too. User cancellations are not. **Error codes parsed from Blob bodies as well as JSON** — a download-typed tool call fails with a Blob, so `errorCodeOf` handles both shapes. **Source attribution for unattended runs** — `sourceId` is threaded from `PolicyRunner` through `PolicyRun` to the recorded row and out to the wire, so a folder, bucket or webhook failure names what fed it. Previously it had none. **Deleting a file closes its failures** — `FileContext.removeFiles` notifies `POST /removed-files`, which transitions those incidents to `FILE_REMOVED`. Terminal, so they leave every reviewer's queue. The rows stay for audit. **The queue can be emptied** — reads now default to open statuses only; ask for a status explicitly to see closed rows. ## Behaviour changes - **Editor failures dedup per person.** `RecordFailure.scopeRef()` includes the actor for TOOL-origin rows, so two people hitting the same failure on the same file are two incidents rather than one. Processor rows are unaffected and their dedup key is byte-identical to before. - **`UNKNOWN` offers only Dismiss.** Acknowledge is no longer offered on it. - **Background reports no longer raise a toast.** Both calls pass `suppressErrorToast`, so a failed report is silent as intended; previously a core build showed the user a "Not Found" toast on every tool failure. ## What is stored File ids only, never names. The request type has no filename field, and a `fileNames` value handed to the client reporter is accepted and ignored. One caveat to review deliberately: the free-text `detail` is stored **verbatim**. `RecordFailure` truncates it at 2000 characters and nothing else; the redaction that used to strip name-shaped text was reverted in `024899f3f6` because it made an unclassified failure impossible to act on. A backend message that embeds a filename (LibreOffice conversion errors, IO errors) will therefore persist that text and show it to a team leader. ## How to test Needs a proprietary or SaaS build with login enabled. `task dev:all` gives you one. 1. **Report a failure from a tool.** Open a PDF, run **Remove Password** on it with a wrong password. Nothing visible changes for you: reporting is silent by design. 2. **See it recorded.** Go to `/processor/documents` and scroll to **Failures** (dev builds only). A row appears titled "Password-protected document", with `Hit by <your user>`. Press **Show raw JSON** to see exactly what was stored. 3. **Confirm no filename is stored as data.** In that JSON, `fileId` is an opaque uuid and there is no name field. Note the `detail` string may contain a filename if the backend put one in its message, per the caveat above. 4. **Confirm the request is capped.** In DevTools, POST to `/api/v1/file-run-events/reports` with 201 entries in `fileIds`. It returns 400 naming the limit, and no rows are added. 5. **Deleting a file clears its failure.** Back in the editor, delete the file you just failed on. Refresh the failures list: its row is gone from the default view. Filter by `FILE_REMOVED` to see it still exists. 6. **Two people, two incidents.** Have a colleague fail the same tool on their own copy of the same file. Two rows, not one occurrence count. ## Migration `source_id` is a new column and `FILE_REMOVED` a new status value. Both are already in the SaaS migration ([Stirling-PDF-SaaS #322](https://github.com/Stirling-Tools/Stirling-PDF-SaaS/pull/322)); self-hosted picks them up from `ddl-auto`. |
||
|
James Brunton
|
6f2b829f72 |
Filter Pipelines page to only show what the user thinks as pipelines (#7495)
# Description of Changes Currently, the Pipelines page shows all backend Policies, which was the desired behaviour when we first designed this, but as it's come along, it doesn't feel right anymore. This adds a filter so the Pipelines table only shows things that have been defined by the user as a New Pipeline, so not Policies etc. ## Before <img width="1510" height="789" alt="image" src="https://github.com/user-attachments/assets/5aebb065-3d42-4483-be3c-253fd8918d49" /> ## After <img width="1512" height="790" alt="image" src="https://github.com/user-attachments/assets/2966a0de-ec9f-43e6-bb1d-c4aeb30dfbf8" /> |
||
|
Anthony Stirling
|
0be10b2dff |
Cucumber concurrency validation plus fix (#7379)
# Description of Changes cucumber tests to run multiple threads of commands at same time --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
Ludy
|
5170509695 |
deps: upgrade mwiede JSch to 2.28.6 and adapt SFTP password handling (#7496)
# Description of Changes This PR replaces #7490 and upgrades `com.github.mwiede:jsch` from `0.2.23` to `2.28.6`. In addition to the dependency bump from the original Dependabot PR, this PR includes the required compatibility adjustment for SFTP password authentication: - Updated `jschVersion` in `build.gradle` from `0.2.23` to `2.28.6`. - Updated `SftpFileClient` to pass the configured password to JSch as UTF-8 encoded bytes instead of using the `String` overload. - Preserved the existing SFTP connection and host-key verification behavior. - Addresses the API compatibility changes introduced by the newer JSch version that prevented the dependency upgrade from being used unchanged. This supersedes #7490 --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
briosandAnthony Stirling
|
4a2329ab6d |
refactor(hibernate): implement manual Hibernate-compliant equals/hashCode for entity classes (#6433)
# Description of Changes This PR refactors our JPA entity classes to replace Lombok's `@Data` and auto-generated `@EqualsAndHashCode` annotations with explicit Lombok annotations and custom, JPA-compliant `equals()` and `hashCode()` implementations. ### Rationale Lombok's default `@Data` and `@EqualsAndHashCode` annotations are not recommended for JPA entities. They often lead to: - Severe performance issues (e.g., loading lazy collections when evaluating `hashCode` or `toString`). - Identity mismatches or collection bugs (e.g., when database-generated IDs transition from `null` to assigned, breaking the entity's lookup in a `Set` or `Map`). This change ensures all JPA entities use safe Hibernate proxy checking and use only the entity's database identifier for equality and hash code calculations. <!-- Please provide a summary of the changes, including: - What was changed - Why the change was made - Any challenges encountered Closes #(issue_number) --> --- ## Checklist ### General - [X] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [X] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [X] I have performed a self-review of my own code - [X] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [X] I have run `task check` to verify linters, typechecks, and tests pass - [X] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. --------- Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com> |
||
|
Ludy
|
a28a950aa4 |
feat: modernize codebase using Java switch expressions and List#getFirst/getLast APIs (#6334)
# Description of Changes This PR introduces a broad modernization of the codebase by adopting newer Java language features and improving code readability and maintainability. ## What was changed - Replaced traditional `switch` statements with modern switch expressions (`case ->`) across multiple classes. - Replaced usages of `List#get(0)` and `List#get(size - 1)` with `getFirst()` and `getLast()` respectively. - Simplified conditional logic using pattern matching (e.g., `instanceof` and switch pattern matching). - Refactored various utility and controller classes to reduce boilerplate and improve clarity. - Removed unused or redundant code (e.g., `parseClientFileIds` method in `MergeController`). - Improved type safety (e.g., using `Class::isInstance` instead of `instanceof` checks in streams). - Cleaned up Spring annotations by removing unnecessary `@Autowired` where constructor injection is already used. - Added a new test (`UIDataControllerTest`) to ensure correct handling of identical JSON configs with different filenames. - Minor formatting and style fixes (e.g., Spotless formatting adjustment). ## Why the change was made - To align the codebase with modern Java standards (Java 17+ features). - To improve readability and maintainability by reducing verbosity. - To eliminate common indexing patterns that are more error-prone. - To standardize coding style across the project. - To improve test coverage for edge cases discovered during refactoring. --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. --------- Signed-off-by: Ludy87 <Ludy87@users.noreply.github.com> |
||
|
Anthony Stirling
|
7a748d4ad2 |
Add stored supporting files for pipeline steps (#7146)
# Description of Changes Backend only change for pipelines to support files (ie pipeline to sign all files with the same cert file etc) --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
Anthony Stirling
|
8b1bfb87f7 |
Add S3 Object Lock retention to policy outputs (#7094)
# Description of Changes Add S3 Object Lock retention to policy outputs (create file and cant be deleted untill after a set deadline passed) --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
ConnorYohandReece Browne
|
df170fd4a6 |
feat(storage): encryption-at-rest ops — audit, admin kill switch, migration, key rotation (PR2) (#7173)
# Description of Changes
**PR2 of the encrypt-at-rest initiative — PR1 was #7155** Makes the P1
crypto operable and compliance-credible: admins can see the feature's
state, flip the kill switch over an API instead of raw SQL, encrypt the
pre-existing plaintext backlog, rotate the master key, and every
security-relevant event lands in the audit trail. No frontend — that's
PR3.
**What was changed**
- **Audit events** — new `STORAGE_ENCRYPTION` audit type, emitted
through a small listener interface so the crypto classes stay plain
objects: `encrypt`, `decrypt` (per-read events honour
`storage.encryption.auditReads`, default **on** — HIPAA reviewers expect
read audit; busy installs can disable), `decrypt.denied` (always),
`key.created/disabled/enabled`, `master.rotated`, `migration.completed`,
plus a `plaintextExport` marker whenever a plaintext copy of
encrypted-at-rest content is served (with `inline` flag to distinguish
in-app view from saved download).
- **Admin API** `/api/v1/admin/storage-encryption` (`hasRole('ADMIN')`):
- `GET /status` — write/decrypt state, **master-key fingerprint**
(SHA-256 prefix for backup verification, never key material), encrypted
vs plaintext file counts, full key list with status history.
- `POST /keys/{id}/disable` / `enable` — the kill switch, now with
active cache invalidation so revocation is immediate on the handling
node (cross-node converges within the 60s cache TTL). Enable is
restricted to DISABLED keys so two ACTIVE keys can't exist per scope.
- `POST /migrate` + `GET /migrate/status` — encrypt-existing job.
- `POST /master/rotate` — key material is never accepted over HTTP; keys
come from config/env.
- **Deliberately no delete endpoint** — key material can be disabled but
never destroyed through the API.
- **Encrypt-existing migration job** — new writes are encrypted from the
moment the flag is on; this converts the backlog. Crash-safe per file:
store the encrypted copy under a NEW storage key → compare-and-swap the
DB row → only then delete the old blob. A CAS miss (user replaced the
file mid-run) discards the job's copy — the user's file always wins.
Worst crash outcome is an orphaned blob, never a lost file; re-runs are
idempotent (`encryption_key_id IS NULL` selection, cursor-paged so
failures can't wedge the loop). Handles all three blobs per row
(main/history/audit-log), runs on a throttled virtual thread,
single-flight guarded.
- **Master-key rotation** — cheap by design thanks to the P1 hierarchy:
rotate re-wraps the handful of KEK rows, zero file I/O. New config
`stirling.security.fileEncryptionKeyPrevious` (+env) gives `unwrap` a
fallback during rotation, and
`stirling.security.fileEncryptionKeyVersion` marks which master wrapped
each row. Runbook: set new key primary + old as previous + bump version
→ restart (startup self-check passes via fallback, warns about pending
rows) → `POST /master/rotate` → remove the previous key.
- **Shared state bean** — `StorageEncryptionState` is built once and
shared by the storage decorator and the admin API, so kill-switch cache
invalidation hits the same caches the decorator reads.
**Reviewer notes**
- The revoked→403 mapping promised for PR2 already landed in #7155 after
manual testing; this PR adds the matching `decrypt.denied` audit event.
- 19 new tests: audit emission (encrypt/decrypt/denied, legacy plaintext
emits nothing), kill-switch immediacy (no TTL wait), rotation
(previous-key fallback, re-wrap + cleanup, idempotent second call),
migration (backlog encrypted byte-identical, CAS-miss discards own copy,
per-file failure counting, concurrent-start rejection, write-disabled
rejection), admin controller status/conflict/not-found paths.
- Full proprietary suite: 2246/2247 green (the one failure is the
pre-existing Windows-symlink FolderIdentitiesTest, unrelated).
---
[ENCRYPTION_AT_REST_TEST_REPORT.html](https://github.com/user-attachments/files/30664158/ENCRYPTION_AT_REST_TEST_REPORT.html)
---------
Co-authored-by: Reece Browne <74901996+reecebrowne@users.noreply.github.com>
|
||
|
EthanHealy01
|
c929386442 |
Record policy-run failures as durable, actionable events (Review Flow PR 1) (#7269)
# Description of Changes PR 1 of the failure-notification work: a durable, team-scoped record of **why a policy run failed**, surfaced in the portal with the triage actions each failure allows. Today a failed policy run is not quite invisible, but it is unusable: the ledger marks the file `ERROR`, and the audit aspect keeps the exception message and status code. Nothing classifies either one, nothing surfaces them, and neither offers a next step. If the file came from a folder, bucket or webhook there is also no user watching, so nobody learns it never made it through. This adds the record and the read surface; the remediation that acts on documents comes later (see below). ## What this does **A failure kind registry as data.** `FailureKind` describes what can go wrong: a stable wire id, i18n keys, an English fallback, and four facets the review surface needs (`Stage`, `Severity`, `Remedy`, `Scope`). It is shaped like the existing `ExceptionUtils.ErrorCode` and *links* to that vocabulary rather than replacing it. **Classification off structured codes, not message matching.** Policy steps dispatch over loopback HTTP, so a tool's 4xx arrives as a `RestClientResponseException` whose body is the Problem Details document carrying `errorCode`. `FailureClassifier` reads that. Anything unrecognised becomes `UNKNOWN`, which is the point: every failed run gets an addressable record from day one, and which kinds to promote next is answered by production frequency rather than guesswork. **Actions declared by a kind, implemented as beans.** A kind lists the `FailureActionId`s it offers; behaviour lives in `FailureAction` beans resolved by id — the idiom this codebase already uses for `InputSource`, `PolicyOutputSink` and `PolicyTrigger`. A kind cannot be sent an action it never declared (400), so an incoherent pairing is unreachable rather than merely unrendered. A new kind ships as a registry entry plus copy: no new endpoint, no UI change. **Repeat folding.** Recording folds a genuine repeat into the existing incident instead of inserting again, keyed on `(team_id, dedup_key)`. That matters for a snapshot-mode source that re-lists every file on each sweep: the same broken file is one incident, not one per sweep. Distinct files keep distinct rows. The unique constraint is enforced by the database, and a writer that loses the insert race folds into the winner's row. One granularity caveat worth naming: nothing populates `file_id` in this PR, so every row has it NULL. A FILE-scoped kind therefore dedups on `policy + run` rather than `policy + file`. That still yields one row per document for the sources shipped here, because the folder, S3 and webhook sources each start one run per file; it stops holding as soon as a single run carries several documents, which is why editor-origin reporting (item 3 below) populates `file_id`. **No document identity is stored.** No file name, no content. `fileId` is an opaque reference only the owner's own client can resolve locally. `detail` keeps the raw message (the only diagnostic an `UNKNOWN` failure has) with anything path- or filename-shaped stripped on the way in, capped at 2,000 characters. `PolicyExecutor`'s type-mismatch message now reports the *extension* rather than the filename, since that message becomes the stored `detail`. **Access.** Reads and triage are leader-only, gated exactly the way `PolicyController` gates policy editing, with the single-user carve-out when login is disabled. Every read and write is scoped to the caller's own team from the authenticated principal — there is no team parameter on the API. Self-hosted needs no migration: the table is created from the entity by `ddl-auto=update`, as with every other table. ## What this does not do yet - **Actions are incident dispositions, not document dispositions.** Acknowledge and Dismiss change how a failure is displayed and touch nothing else — not the document, not the processed-file ledger, not the run, not any output destination. That is what makes them safe to offer against `UNKNOWN`, and why there is no Approve/Release yet. - **Two kinds only.** `INPUT_PASSWORD_PROTECTED` and `UNKNOWN`. Everything else classifies as `UNKNOWN` and shows its raw message. - **Editor-origin failures are not reported.** Every row is `PROCESSOR`. `FailureOrigin.EDITOR` and `API` exist in the enum but nothing writes them. - **The list is dev-only for now.** The section renders behind `import.meta.env.DEV`, so it ships in no production bundle. The endpoints are live and gated. - **No retention or per-team cap** on `file_run_events`. Tracked separately. - **No suspend-and-prompt.** `PolicyInputRequiredException` and the engine's `suspend()` exist but nothing throws it, so a run cannot pause to ask for a password today. - **SaaS needs a migration** in `Stirling-PDF-SaaS` (`CREATE TABLE IF NOT EXISTS stirling_pdf.file_run_events`), per the convention documented at `app/saas/src/main/resources/application-saas.properties:21`. ## What follows in later PRs 1. **Map the remaining error codes to specific kinds** — corrupted file, OCR unavailable, output destination unreachable, entitlement refusals, and so on — each with its own copy and its own action set, replacing today's `UNKNOWN` catch-all with a named notification in the review UI. 2. **Real remediation actions** attached to those kinds: fix (supply a password and resume), skip (drop this file, continue the batch), and decline (reject an incoming file outright), acting on the held document rather than only on the incident row. This is where the suspend-and-prompt path gets wired. 3. **Editor-origin reporting**, so a failure a user hits in the editor lands in the same queue as one from a bucket. 4. **The user-facing review surface**: notifications with a sticky review section, per-file badges, and an export gate, with the dev-only list here replaced by the real thing. ## How to test Needs a SaaS or proprietary build with login enabled, and an account that leads a team. 1. Create a policy in the Processor with any step (Auto-redact is fine) and a source you can drop files into. 2. Upload two files that will fail it: **a password-protected PDF**, and **a corrupted PDF** (truncate a valid one, or rename a `.csv` to `.pdf`). 3. Let the policy run and fail on both. 4. Go to the portal's **Documents** view and scroll to **Failures** (dev builds only). Expect two rows: - **Password-protected document** — classified from `E004`, with the kind's own labels **"I'll unlock this"** and **"Skip this file"** rather than generic wording. - **Unrecognised failure** — the corrupted file, classified `UNKNOWN` (`E001` is not claimed by a kind yet), showing its raw message with generic **Acknowledge** / **Dismiss**. Neither row contains a file name anywhere, including in the raw message. Press **Show raw JSON** to read exactly what the server returned. Acting on a row transitions it and comes back with both buttons disabled and a reason. Re-running the same batch increments the occurrence count on the existing rows rather than adding new ones; two *different* password-protected files produce two separate rows. --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass |
||
|
Anthony Stirling
|
2cf6db99ce |
Fix timing-fragile Valkey rate-limit boundary test (#7302)
# Description of Changes Fix timing-fragile Valkey rate-limit boundary test --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
Ludy
|
b10fc1b2de |
fix(java): prevent executor, task, regex, and stream resource leaks (#7284)
# Description of Changes - Added graceful shutdown handling for service-owned executors in `JobExecutorService`, `PolicyEngine`, and `AsyncConfig`. - Added expiration and cleanup for abandoned pending jobs in `TaskManager`. - Replaced the unbounded regex pattern cache with a bounded cache limited to 512 entries. - Ensured `Files.walk()` is closed correctly in `MobileScannerService`. - These changes prevent unbounded heap growth, lingering virtual-thread executors, and file-descriptor leaks. - Added configurable pending-job expiration through `stirling.job.pendingExpiryMinutes`, defaulting to 24 hours. --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
Ludy
|
866e56728d |
fix(storage): delete share access records before expired share links (#7161)
# Description of Changes - Updated expired share-link cleanup to delete related `FileShareAccess` records before deleting their parent `FileShare` records. - Wrapped the cleanup operation in a transaction to ensure the deletion order is enforced atomically. - Prevents foreign-key constraint violations and scheduled-task failures during cleanup. - The full backend check was limited by a Gradle distribution download/network error. ```cmd [backend:dev:proprietary] 16:25:43.362 [scheduled-vt-2] WARN org.hibernate.orm.jdbc.error - HHH000247: ErrorCode: 23503, SQLState: 23503 [backend:dev:proprietary] 16:25:43.362 [scheduled-vt-2] WARN org.hibernate.orm.jdbc.error - Referentielle Integrität verletzt: "FKQ6V4QH5LFCAWII0ABRVSJO5SG: PUBLIC.FILE_SHARE_ACCESSES FOREIGN KEY(FILE_SHARE_ID) REFERENCES PUBLIC.FILE_SHARES(FILE_SHARE_ID) (CAST(97 AS BIGINT))" [backend:dev:proprietary] Referential integrity constraint violation: "FKQ6V4QH5LFCAWII0ABRVSJO5SG: PUBLIC.FILE_SHARE_ACCESSES FOREIGN KEY(FILE_SHARE_ID) REFERENCES PUBLIC.FILE_SHARES(FILE_SHARE_ID) (CAST(97 AS BIGINT))"; SQL statement: [backend:dev:proprietary] delete from file_shares where file_share_id=? [23503-240] [backend:dev:proprietary] 16:25:43.380 [scheduled-vt-2] ERROR o.s.s.s.TaskUtils$LoggingErrorHandler - Unexpected error occurred in scheduled task [backend:dev:proprietary] org.springframework.dao.DataIntegrityViolationException: could not execute statement [Referentielle Integrität verletzt: "FKQ6V4QH5LFCAWII0ABRVSJO5SG: PUBLIC.FILE_SHARE_ACCESSES FOREIGN KEY(FILE_SHARE_ID) REFERENCES PUBLIC.FILE_SHARES(FILE_SHARE_ID) (CAST(97 AS BIGINT))" [backend:dev:proprietary] Referential integrity constraint violation: "FKQ6V4QH5LFCAWII0ABRVSJO5SG: PUBLIC.FILE_SHARE_ACCESSES FOREIGN KEY(FILE_SHARE_ID) REFERENCES PUBLIC.FILE_SHARES(FILE_SHARE_ID) (CAST(97 AS BIGINT))"; SQL statement: [backend:dev:proprietary] delete from file_shares where file_share_id=? [23503-240]] [delete from file_shares where file_share_id=?]; SQL [delete from file_shares where file_share_id=?]; constraint [FKQ6V4QH5LFCAWII0ABRVSJO5SG] [backend:dev:proprietary] at org.springframework.orm.jpa.hibernate.HibernateExceptionTranslator.convertHibernateAccessException(HibernateExceptionTranslator.java:169) [backend:dev:proprietary] at org.springframework.orm.jpa.hibernate.HibernateExceptionTranslator.convertHibernateAccessException(HibernateExceptionTranslator.java:131) [backend:dev:proprietary] at org.springframework.orm.jpa.hibernate.HibernateExceptionTranslator.translateExceptionIfPossible(HibernateExceptionTranslator.java:105) [backend:dev:proprietary] at org.springframework.orm.jpa.vendor.HibernateJpaDialect.translateExceptionIfPossible(HibernateJpaDialect.java:223) [backend:dev:proprietary] at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:557) [backend:dev:proprietary] at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:794) [backend:dev:proprietary] at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:757) [backend:dev:proprietary] at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:687) [backend:dev:proprietary] at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:408) [backend:dev:proprietary] at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:130) [backend:dev:proprietary] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) [backend:dev:proprietary] at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:135) [backend:dev:proprietary] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) [backend:dev:proprietary] at org.springframework.data.jpa.repository.support.CrudMethodMetadataPostProcessor$CrudMethodMetadataPopulatingMethodInterceptor.invoke(CrudMethodMetadataPostProcessor.java:166) [backend:dev:proprietary] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) [backend:dev:proprietary] at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:222) [backend:dev:proprietary] at jdk.proxy4/jdk.proxy4.$Proxy246.deleteAll(Unknown Source) [backend:dev:proprietary] at stirling.software.proprietary.storage.service.StorageCleanupService.cleanupExpiredShareLinks(StorageCleanupService.java:71) [backend:dev:proprietary] at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104) [backend:dev:proprietary] at java.base/java.lang.reflect.Method.invoke(Method.java:565) [backend:dev:proprietary] at org.springframework.scheduling.support.ScheduledMethodRunnable.runInternal(ScheduledMethodRunnable.java:128) [backend:dev:proprietary] at org.springframework.scheduling.support.ScheduledMethodRunnable.lambda$run$1(ScheduledMethodRunnable.java:122) [backend:dev:proprietary] at io.micrometer.observation.Observation.observe(Observation.java:569) [backend:dev:proprietary] at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:122) [backend:dev:proprietary] at org.springframework.scheduling.config.Task$OutcomeTrackingRunnable.run(Task.java:88) [backend:dev:proprietary] at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) [backend:dev:proprietary] at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:545) [backend:dev:proprietary] at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:369) [backend:dev:proprietary] at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:310) [backend:dev:proprietary] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1090) [backend:dev:proprietary] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:614) [backend:dev:proprietary] at java.base/java.lang.VirtualThread.run(VirtualThread.java:460) [backend:dev:proprietary] Caused by: org.hibernate.exception.ConstraintViolationException: could not execute statement [Referentielle Integrität verletzt: "FKQ6V4QH5LFCAWII0ABRVSJO5SG: PUBLIC.FILE_SHARE_ACCESSES FOREIGN KEY(FILE_SHARE_ID) REFERENCES PUBLIC.FILE_SHARES(FILE_SHARE_ID) (CAST(97 AS BIGINT))" [backend:dev:proprietary] Referential integrity constraint violation: "FKQ6V4QH5LFCAWII0ABRVSJO5SG: PUBLIC.FILE_SHARE_ACCESSES FOREIGN KEY(FILE_SHARE_ID) REFERENCES PUBLIC.FILE_SHARES(FILE_SHARE_ID) (CAST(97 AS BIGINT))"; SQL statement: [backend:dev:proprietary] delete from file_shares where file_share_id=? [23503-240]] [delete from file_shares where file_share_id=?] [backend:dev:proprietary] at org.hibernate.dialect.H2Dialect.lambda$buildSQLExceptionConversionDelegate$0(H2Dialect.java:840) [backend:dev:proprietary] at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:34) [backend:dev:proprietary] at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:115) [backend:dev:proprietary] at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:184) [backend:dev:proprietary] at org.hibernate.engine.jdbc.mutation.internal.AbstractMutationExecutor.performNonBatchedMutation(AbstractMutationExecutor.java:145) [backend:dev:proprietary] at org.hibernate.engine.jdbc.mutation.internal.MutationExecutorSingleNonBatched.performNonBatchedOperations(MutationExecutorSingleNonBatched.java:53) [backend:dev:proprietary] at org.hibernate.engine.jdbc.mutation.internal.AbstractMutationExecutor.execute(AbstractMutationExecutor.java:66) [backend:dev:proprietary] at org.hibernate.persister.entity.mutation.AbstractDeleteCoordinator.doStaticDelete(AbstractDeleteCoordinator.java:268) [backend:dev:proprietary] at org.hibernate.persister.entity.mutation.AbstractDeleteCoordinator.delete(AbstractDeleteCoordinator.java:79) [backend:dev:proprietary] at org.hibernate.action.internal.EntityDeleteAction.execute(EntityDeleteAction.java:119) [backend:dev:proprietary] at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:634) [backend:dev:proprietary] at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:505) [backend:dev:proprietary] at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:381) [backend:dev:proprietary] at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:40) [backend:dev:proprietary] at org.hibernate.event.service.internal.EventListenerGroupImpl.fireEventOnEachListener(EventListenerGroupImpl.java:138) [backend:dev:proprietary] at org.hibernate.internal.SessionImpl.fireFlush(SessionImpl.java:1484) [backend:dev:proprietary] at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:481) [backend:dev:proprietary] at org.hibernate.internal.SessionImpl.flushBeforeTransactionCompletion(SessionImpl.java:2111) [backend:dev:proprietary] at org.hibernate.internal.SessionImpl.beforeTransactionCompletion(SessionImpl.java:2033) [backend:dev:proprietary] at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.beforeTransactionCompletion(JdbcCoordinatorImpl.java:410) [backend:dev:proprietary] at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.beforeCompletionCallback(JdbcResourceLocalTransactionCoordinatorImpl.java:166) [backend:dev:proprietary] at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commitNoRollbackOnly(JdbcResourceLocalTransactionCoordinatorImpl.java:248) [backend:dev:proprietary] at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commit(JdbcResourceLocalTransactionCoordinatorImpl.java:242) [backend:dev:proprietary] at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:89) [backend:dev:proprietary] at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:553) [backend:dev:proprietary] ... 27 common frames omitted [backend:dev:proprietary] Caused by: org.h2.jdbc.JdbcSQLIntegrityConstraintViolationException: Referentielle Integrität verletzt: "FKQ6V4QH5LFCAWII0ABRVSJO5SG: PUBLIC.FILE_SHARE_ACCESSES FOREIGN KEY(FILE_SHARE_ID) REFERENCES PUBLIC.FILE_SHARES(FILE_SHARE_ID) (CAST(97 AS BIGINT))" [backend:dev:proprietary] Referential integrity constraint violation: "FKQ6V4QH5LFCAWII0ABRVSJO5SG: PUBLIC.FILE_SHARE_ACCESSES FOREIGN KEY(FILE_SHARE_ID) REFERENCES PUBLIC.FILE_SHARES(FILE_SHARE_ID) (CAST(97 AS BIGINT))"; SQL statement: [backend:dev:proprietary] delete from file_shares where file_share_id=? [23503-240] [backend:dev:proprietary] at org.h2.message.DbException.getJdbcSQLException(DbException.java:520) [backend:dev:proprietary] at org.h2.message.DbException.getJdbcSQLException(DbException.java:489) [backend:dev:proprietary] at org.h2.message.DbException.get(DbException.java:223) [backend:dev:proprietary] at org.h2.message.DbException.get(DbException.java:199) [backend:dev:proprietary] at org.h2.constraint.ConstraintReferential.checkRow(ConstraintReferential.java:363) [backend:dev:proprietary] at org.h2.constraint.ConstraintReferential.checkRowRefTable(ConstraintReferential.java:380) [backend:dev:proprietary] at org.h2.constraint.ConstraintReferential.checkRow(ConstraintReferential.java:254) [backend:dev:proprietary] at org.h2.table.Table.fireConstraints(Table.java:1208) [backend:dev:proprietary] at org.h2.table.Table.fireAfterRow(Table.java:1226) [backend:dev:proprietary] at org.h2.command.dml.Delete.update(Delete.java:81) [backend:dev:proprietary] at org.h2.command.dml.DataChangeStatement.update(DataChangeStatement.java:77) [backend:dev:proprietary] at org.h2.command.CommandContainer.update(CommandContainer.java:139) [backend:dev:proprietary] at org.h2.command.Command.executeUpdate(Command.java:306) [backend:dev:proprietary] at org.h2.command.Command.executeUpdate(Command.java:250) [backend:dev:proprietary] at org.h2.jdbc.JdbcPreparedStatement.executeUpdateInternal(JdbcPreparedStatement.java:213) [backend:dev:proprietary] at org.h2.jdbc.JdbcPreparedStatement.executeUpdate(JdbcPreparedStatement.java:172) [backend:dev:proprietary] at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeUpdate(ProxyPreparedStatement.java:61) [backend:dev:proprietary] at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeUpdate(HikariProxyPreparedStatement.java) [backend:dev:proprietary] at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:181) [backend:dev:proprietary] ... 48 common frames omitted ``` --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
brios
|
a2dd0298dc |
refactor(api): replace length checks with isEmpty (#7214)
# Description of Changes Stylistic problem reported by static analyzer. Changes: * Replaced `sb.length() > 0` and `sb.length() == 0` with `!sb.isEmpty()` and `sb.isEmpty()` for `StringBuilder`, `String`, and collections throughout the codebase, improving readability and aligning with modern Java best practices. <!-- Please provide a summary of the changes, including: - What was changed - Why the change was made - Any challenges encountered Closes #(issue_number) --> --- ## Checklist ### General - [x] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [x] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [x] I have performed a self-review of my own code - [x] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [x] I have run `task check` to verify linters, typechecks, and tests pass - [x] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
ConnorYoh
|
50bc4a7866 |
fix(storage): don't query the encryption key registry when storage is off (unblocks backend:dev:saas) (#7265)
# Description of Changes Fixes a startup failure introduced by #7155 and reported against `task backend:dev:saas`. **What goes wrong** `StorageProviderConfig.storageEncryptionState(...)` is created on every startup, in every profile. When `storage.encryption.enabled` is false — the default, and what SaaS ships — the `||` short-circuit evaluates `fileEncryptionKeyRepository.count()`, a live query against `file_encryption_keys`: ```java if (writeEnabled || fileEncryptionKeyRepository.count() > 0) { // <- always runs when the flag is off ``` That table only exists if `ddl-auto=update` managed to create it. When it cannot — permissions on a shared Supabase branch DB, concurrent DDL from several developers, schema ordering — **ddl-auto logs and continues**, so the situation used to be a warning nobody noticed. Now it is a query that throws during bean creation and takes the whole context down. Two things make this sting in SaaS specifically: `storage.enabled` is false there, so before this feature nothing ever touched the table; and `hibernate.default_schema=stirling_pdf` means the table has to exist in a schema the app may not be able to create in. There is a second exposure on the request path: `suppressDirectDownloads()` also counts (60s cached), so even a surviving boot could 500 on downloads. **Fix** - The boot probe runs only when `storage.enabled` is true, so a deployment that does not use storage never touches the table. - Registry reads are wrapped. The boot probe degrades to "no keys" rather than propagating; `suppressDirectDownloads()` **fails safe by suppressing** rather than issuing a presigned URL it cannot vouch for. Losing the direct-download fast path is recoverable; serving ciphertext is not. **Safety is unchanged, and that is the important part.** The decorator is still installed unconditionally, so any blob carrying the `SPDFEAR1` magic is still decrypted via lazy materialisation or fails loudly — the eager probe only ever bought *earlier* master-key verification. A node that can actually serve stored files has `storage.enabled` on by definition, which is exactly the node the drifted-node protection is for; that test now configures it that way, and a new test pins that the decorator remains installed even with storage off. **Tests** — storage-disabled never calls `count()`; an unreadable registry still boots *and* still suppresses direct downloads; the decorator stays installed with storage off; storage-enabled still probes. Full proprietary suite green apart from the pre-existing Windows-symlink `FolderIdentitiesTest` failure, which is environmental and unrelated. **Note on scope:** deliberately minimal so it can land quickly. The Aikido `findAll()` code-quality finding lives in #7173 only (`rotateMasterKey` does not exist on main), so it is fixed there rather than here. #7173 will be rebased once this merges. --- ## Checklist ### General - [x] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [x] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [x] I have performed a self-review of my own code - [x] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [x] I have run `task check` to verify linters, typechecks, and tests pass - [x] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
Anthony Stirling
|
21dff695fe |
Add SFTP, FTP and SMB network sources to the processor (#7153)
# Description of Changes Add SFTP, FTP and SMB network sources to the processor plus UI change to enable it --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
James BruntonandAnthony Stirling
|
cd199c8659 |
Define tool inputs & outputs in a structured way (#7204)
# Description of Changes Change tool APIs to use structured definitions for input/output/type info because we need that info to be able to validate whether policies can actually successfully work based on whether one tool accepts the output of another. There were various bugs in the previous string definitions because of either misspellings or just incorrect definitions, so I've gone through and fixed all that I can find. <img width="729" height="271" alt="image" src="https://github.com/user-attachments/assets/08357e96-6fbb-4b9c-ba4d-8995420c7b86" /> <img width="749" height="264" alt="image" src="https://github.com/user-attachments/assets/76f46284-1866-4b64-b1ed-2480e01866e9" /> <img width="402" height="636" alt="image" src="https://github.com/user-attachments/assets/8f7a36ca-2845-4f14-a2df-ec9c772e66f6" /> <img width="393" height="317" alt="image" src="https://github.com/user-attachments/assets/46d8b891-9820-4ce3-8109-a8b782277037" /> --------- Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com> |
||
|
dependabot[bot]
|
88cdfb3a43 |
build(deps): bump org.postgresql:postgresql from 42.7.11 to 42.7.13 (#7243)
Bumps [org.postgresql:postgresql](https://github.com/pgjdbc/pgjdbc) from 42.7.11 to 42.7.13. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/pgjdbc/pgjdbc/releases">org.postgresql:postgresql's releases</a>.</em></p> <blockquote> <h2>v42.7.13</h2> <h2>Changes</h2> <ul> <li>docs: add 42.7.13 release changelog <a href="https://github.com/davecramer"><code>@davecramer</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4270">#4270</a>)</li> <li>Adjust EditorConfig für Makefile <a href="https://github.com/BaumiCoder"><code>@BaumiCoder</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4279">#4279</a>)</li> <li>fix(scram): fail closed on channel-binding downgrade (no scram bump) <a href="https://github.com/vlsi"><code>@vlsi</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4272">#4272</a>)</li> <li>Bump pgjdbc version from 42.7.12 to 42.7.13 <a href="https://github.com/davecramer"><code>@davecramer</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4269">#4269</a>)</li> <li>chore: remove test-anorm-sbt module and its disabled CI wiring <a href="https://github.com/vlsi"><code>@vlsi</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4261">#4261</a>)</li> <li>refactor(test-gss): convert to Java/JUnit 5 submodule of the main build <a href="https://github.com/vlsi"><code>@vlsi</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4166">#4166</a>)</li> <li>ci: derive PG test versions from a Renovate-managed maxPgVersion <a href="https://github.com/vlsi"><code>@vlsi</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4218">#4218</a>)</li> <li>feat(insert): cap reWriteBatchedInserts by the protocol limit, not 128 <a href="https://github.com/vlsi"><code>@vlsi</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4207">#4207</a>)</li> <li>refactor(metadata): derive getPrimaryKeys from pg_constraint.conkey <a href="https://github.com/vlsi"><code>@vlsi</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4202">#4202</a>)</li> <li>fix(protocol): defer flushes until response processing <a href="https://github.com/vlsi"><code>@vlsi</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4196">#4196</a>)</li> <li>fix(build): resolve the Temurin 8 test toolchain by vendor <a href="https://github.com/vlsi"><code>@vlsi</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4257">#4257</a>)</li> <li>build: include multi-release source sets in the JaCoCo coverage report <a href="https://github.com/vlsi"><code>@vlsi</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4256">#4256</a>)</li> <li>fix(ci): read java_vendor before overwriting java_distribution <a href="https://github.com/vlsi"><code>@vlsi</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4255">#4255</a>)</li> <li>ci: generate the whole matrix in one batch, coverage job included <a href="https://github.com/vlsi"><code>@vlsi</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4253">#4253</a>)</li> <li>ci: pass CODECOV_TOKEN so protected-branch coverage uploads succeed <a href="https://github.com/vlsi"><code>@vlsi</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4254">#4254</a>)</li> <li>ci: collect coverage on one pinned job <a href="https://github.com/vlsi"><code>@vlsi</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4245">#4245</a>)</li> <li>ci: apply -DqueryTimeout from the matrix query_timeout axis <a href="https://github.com/vlsi"><code>@vlsi</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4246">#4246</a>)</li> <li>ci: make Codecov project and patch statuses informational <a href="https://github.com/vlsi"><code>@vlsi</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4244">#4244</a>)</li> <li>fix(build): restore JaCoCo XML report so Codecov receives coverage <a href="https://github.com/vlsi"><code>@vlsi</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4240">#4240</a>)</li> <li>test(replication): shrink big-transaction inserts to avoid CI timeouts <a href="https://github.com/vlsi"><code>@vlsi</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4243">#4243</a>)</li> <li>update maintainers <a href="https://github.com/davecramer"><code>@davecramer</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4222">#4222</a>)</li> <li>test: add hermetic test for localSocketAddress <a href="https://github.com/vlsi"><code>@vlsi</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4224">#4224</a>)</li> <li>docs(translation): clean up leftover German header in ja.po <a href="https://github.com/vlsi"><code>@vlsi</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4206">#4206</a>)</li> <li>Update ja.po <a href="https://github.com/davecramer"><code>@davecramer</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/2004">#2004</a>)</li> <li>test: add PostgreSQL 18 to the CI test matrix <a href="https://github.com/vlsi"><code>@vlsi</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4198">#4198</a>)</li> <li>test: silence expected SSPI warning stack trace in SSPIClientWaffleTest <a href="https://github.com/vlsi"><code>@vlsi</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4197">#4197</a>)</li> <li>fix(ssl): build PKIX trust anchors without a KeyStore so FIPS-mode JVMs can load sslrootcert <a href="https://github.com/vlsi"><code>@vlsi</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4193">#4193</a>)</li> <li>test: fix flaky sentLocationEqualToLastReceiveLSN replication test <a href="https://github.com/vlsi"><code>@vlsi</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4175">#4175</a>)</li> <li>build: promote MethodCanBeStatic to error level <a href="https://github.com/vlsi"><code>@vlsi</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4172">#4172</a>)</li> <li>Fix PGInterval.setSeconds to reject out of range and NaN values <a href="https://github.com/sehrope"><code>@sehrope</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4194">#4194</a>)</li> <li>Replace connectThreadFactory with connectExecutor <a href="https://github.com/sehrope"><code>@sehrope</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4165">#4165</a>)</li> <li>Fix deleting temp file when spooling large stream to disk in StreamWrapper <a href="https://github.com/sehrope"><code>@sehrope</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4190">#4190</a>)</li> <li>chore: Add top level /scratch to gitignore <a href="https://github.com/sehrope"><code>@sehrope</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4164">#4164</a>)</li> <li>refactor: favour composition over inheritance for Driver.ConnectTask <a href="https://github.com/vlsi"><code>@vlsi</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4160">#4160</a>)</li> <li>Fix NumberParser.getFastLong(...) handling of overlong values <a href="https://github.com/sehrope"><code>@sehrope</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4163">#4163</a>)</li> <li>build: produce a multi-release jar from reduced-pom.xml on Java 11+ <a href="https://github.com/vlsi"><code>@vlsi</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4157">#4157</a>)</li> <li>Add connectThreadFactory and refactor Driver to use FutureTask for loginTimeout connection attempts <a href="https://github.com/sehrope"><code>@sehrope</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4120">#4120</a>)</li> <li>test: verify custom properties reach socket factory <a href="https://github.com/vlsi"><code>@vlsi</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4125">#4125</a>)</li> <li>test: fix LazyCleanerTest timeouts for the lingering Java 8 cleanup thread <a href="https://github.com/vlsi"><code>@vlsi</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4122">#4122</a>)</li> <li>test: stabilise StatementTest.fastCloses on Windows <a href="https://github.com/vlsi"><code>@vlsi</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4121">#4121</a>)</li> <li>fix: append default non-proxy hosts when socksNonProxyHosts is set <a href="https://github.com/davecramer"><code>@davecramer</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4045">#4045</a>)</li> <li>test: budget terminating Sync in BatchDeadlockTest small-RETURNING branch <a href="https://github.com/vlsi"><code>@vlsi</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4116">#4116</a>)</li> <li>test: make message assertions locale-independent <a href="https://github.com/vlsi"><code>@vlsi</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4113">#4113</a>)</li> <li>build: drop xgettext default keywords; regenerate translations <a href="https://github.com/vlsi"><code>@vlsi</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4100">#4100</a>)</li> <li>ci: opt-in scheduled workflows via ENABLE_SCHEDULED_JOBS repo variable <a href="https://github.com/vlsi"><code>@vlsi</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4085">#4085</a>)</li> <li>Avoid direct java.lang.management dependency in maxResultBuffer parser <a href="https://github.com/mblakley-casana"><code>@mblakley-casana</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4069">#4069</a>)</li> <li>fix: restore pre-describe for generated-key batches <a href="https://github.com/bilalshehata"><code>@bilalshehata</code></a> (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4014">#4014</a>)</li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/pgjdbc/pgjdbc/blob/master/CHANGELOG.md">org.postgresql:postgresql's changelog</a>.</em></p> <blockquote> <h2>[42.7.13] (2026-07-06)</h2> <h3>Added</h3> <ul> <li>feat: invalidate the prepared-statement cache when the server reports a <code>search_path</code> change via GUC_REPORT (PostgreSQL 18+), so cached plans are no longer used against the wrong schema [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4259">#4259</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4259">pgjdbc/pgjdbc#4259</a>)</li> <li>feat: <code>reWriteBatchedInserts</code> now merges up to 32768 rows into one multi-values <code>INSERT</code> (bounded by the 65535 bind-parameter limit on the extended protocol) instead of capping at 128, which speeds up batches of few-column rows. The new <code>reWriteBatchedInsertsSize</code> connection property lowers that cap when set; the default of <code>0</code> uses that maximum. [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4207">#4207</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4207">pgjdbc/pgjdbc#4207</a>)</li> <li>feat: invalidate the prepared-statement cache after CREATE/DROP/ALTER so callers no longer trip on "cached plan must not change result type" without opting into <code>autosave=ALWAYS</code>. Controlled by the new <code>flushCacheOnDdl</code> connection property (default <code>true</code>); set to <code>false</code> for the prior behaviour. [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4067">#4067</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4067">pgjdbc/pgjdbc#4067</a>)</li> <li>feat: add <code>connectExecutor</code> connection property to customize the <code>Executor</code> used to run the worker task that performs the connection attempt when <code>loginTimeout</code> is in effect. The value is the fully qualified name of a class implementing <code>java.util.concurrent.Executor</code>. With a null value, the default, the driver retains the prior behavior of running the connection attempt on a daemon thread named <code>"PostgreSQL JDBC driver connection thread"</code>. The executor must run the task on a thread other than the caller's. Running the attempt on a named thread lets applications that monitor driver-created threads identify it. [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4165">#4165</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4165">pgjdbc/pgjdbc#4165</a>)</li> <li>feat: add <code>classLoaderStrategy</code> connection property to control which classloaders the driver searches when loading a class named by a connection property, for example <code>socketFactory</code>. The default <code>driver-first</code> now falls back to the thread context classloader when the driver's classloader cannot resolve the class, which fixes class loading in non-flat class paths such as Quarkus and OSGi. Set <code>driver</code> to keep the previous driver-classloader-only behaviour, or <code>context-first</code> to prefer the thread context classloader [Issue <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/2112">#2112</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/2112">pgjdbc/pgjdbc#2112</a>) [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4167">#4167</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4167">pgjdbc/pgjdbc#4167</a>)</li> <li>feat: add OID constants for geometric arrays, <code>RECORD</code>, and <code>refcursor</code> [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4220">#4220</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4220">pgjdbc/pgjdbc#4220</a>)</li> <li>feat: <code>LargeObject</code> <code>BlobInputStream</code> now skips by seeking instead of reading, and the driver exposes the server version so it can select the 64-bit large-object API where available [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4204">#4204</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4204">pgjdbc/pgjdbc#4204</a>)</li> </ul> <h3>Changed</h3> <ul> <li>refactor: the worker that runs the connection attempt under <code>loginTimeout</code> is now a <code>FutureTask</code> (<code>ConnectTask</code>) instead of the hand-rolled <code>ConnectThread</code>. When the caller hits the timeout, the task is now cancelled with <code>cancel(true)</code>, which interrupts the worker thread rather than letting it run to completion. This makes the connection attempt interruptible, so <code>loginTimeout</code> can stop a slow connection attempt instead of leaking a thread. As before, a connection that the worker still manages to establish after the caller gives up is closed by the worker so that it does not leak. There are no public API changes and this should only lead to faster background resource cleanup for connections that time out. [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4120">#4120</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4120">pgjdbc/pgjdbc#4120</a>)</li> <li>chore: <code>PGXAConnection.ConnectionHandler</code> now rejects <code>setAutoCommit(false)</code> and <code>setSavepoint(...)</code> during an active XA branch, in addition to the long-rejected <code>setAutoCommit(true)</code> / <code>commit()</code> / <code>rollback()</code>. The <code>setSavepoint</code> rejection was already meant to be in place but the guard misspelled the method name as <code>setSavePoint</code>, so savepoints silently went through. Both changes bring the proxy in line with JTA 1.2 §3.4. [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4114">#4114</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4114">pgjdbc/pgjdbc#4114</a>)</li> <li>chore: <code>commitPrepared</code> / <code>rollback</code>-of-prepared now return <code>XAER_RMFAIL</code> instead of <code>XAER_RMERR</code> when the underlying connection is left in a non-idle <code>TransactionState</code>. Transaction managers (Geronimo, Narayana, Atomikos) treat <code>XAER_RMFAIL</code> as retryable on a fresh <code>XAResource</code>; the prepared transaction is no longer abandoned. [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4114">#4114</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4114">pgjdbc/pgjdbc#4114</a>)</li> <li>refactor: derive <code>getPrimaryKeys</code> from <code>pg_constraint.conkey</code> [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4202">#4202</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4202">pgjdbc/pgjdbc#4202</a>)</li> </ul> <h3>Fixed</h3> <ul> <li>fix: the published GitHub release now ships the released <code>postgresql-<version>.jar</code> and its detached PGP signature, taken from the same signed build that is uploaded to Maven Central, instead of a leftover SNAPSHOT jar [Issue <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/3812">#3812</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/3812">pgjdbc/pgjdbc#3812</a>) [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/3814">#3814</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/3814">pgjdbc/pgjdbc#3814</a>)</li> <li>fix: simplify the <code>Statement#cancel</code> state machine by dropping the redundant <code>CANCELLED</code> state. <code>killTimerTask</code> now waits for the state to return to <code>IDLE</code> directly, which removes a spin-forever case when more than one thread observes the cancel completing [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/1827">#1827</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/1827">pgjdbc/pgjdbc#1827</a>).</li> <li>perf: defer simple-query flushes until the driver reads the response, allowing <code>BEGIN</code> and the following query to share a network flush [Issue <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/3894">#3894</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/3894">pgjdbc/pgjdbc#3894</a>) [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4196">#4196</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4196">pgjdbc/pgjdbc#4196</a>)</li> <li>fix: <code>reWriteBatchedInserts</code> no longer throws <code>IllegalArgumentException</code> when batching a parameterless <code>INSERT</code> (for example <code>INSERT INTO t VALUES (1, 2)</code>) of 256 rows or more [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4207">#4207</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4207">pgjdbc/pgjdbc#4207</a>)</li> <li>fix: a comment before <code>CALL</code> in a <code>CallableStatement</code> no longer hides the native call, so OUT parameter registration works for <code>/* comment */ call proc(?, ?)</code> and similar. <code>Parser.modifyJdbcCall</code> now skips leading whitespace and SQL comments (both <code>--</code> and <code>/* */</code>) before the call, tolerates a trailing comment after a <code>{ ... }</code> escape, and no longer adds a spurious comma when moving an OUT parameter into a call whose arguments are only a comment [Issue <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/2538">#2538</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/2538">pgjdbc/pgjdbc#2538</a>) [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4209">#4209</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4209">pgjdbc/pgjdbc#4209</a>)</li> <li>fix: <code>PreparedStatement.toString()</code> no longer throws for a <code>bytea</code> value supplied as text via <code>PGobject</code>. Hex-format values (<code>\x...</code>) are validated and rendered as a <code>bytea</code> literal, and escape-format values are quoted and cast like any other literal [Issue <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/3757">#3757</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/3757">pgjdbc/pgjdbc#3757</a>) [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4201">#4201</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4201">pgjdbc/pgjdbc#4201</a>)</li> <li>fix: the driver no longer nulls the <code>contextClassLoader</code> of shared <code>ForkJoinPool.commonPool()</code> worker threads, which previously left unrelated tasks on those threads running with a <code>null</code> classloader [Issue <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4155">#4155</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4155">pgjdbc/pgjdbc#4155</a>) [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4156">#4156</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4156">pgjdbc/pgjdbc#4156</a>)</li> <li>fix: <code>PgResultSet#getCharacterStream</code> wraps <code>String</code> in a <code>StringReader</code> [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4063">#4063</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4063">pgjdbc/pgjdbc#4063</a>)</li> <li>fix: <code>PGXAConnection</code> no longer saves and restores the underlying connection's JDBC <code>autoCommit</code> flag. All XA-protocol SQL (<code>BEGIN</code>, <code>PREPARE TRANSACTION</code>, <code>COMMIT</code>, <code>ROLLBACK</code>, <code>COMMIT PREPARED</code>, <code>ROLLBACK PREPARED</code>, the <code>recover()</code> SELECT) is sent through <code>QUERY_SUPPRESS_BEGIN</code>, so the caller's <code>autoCommit</code> value is invariant across every <code>XAResource</code> call. Fixes the "2nd phase commit must be issued using an idle connection" failure during recovery on managed datasources that pool connections with <code>autoCommit=false</code> (TomEE, WildFly, WebSphere Liberty) [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4114">#4114</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4114">pgjdbc/pgjdbc#4114</a>)</li> <li>fix: <code>PGXAConnection.prepare()</code> now mutates XA state only after <code>PREPARE TRANSACTION</code> succeeds. A failed <code>PREPARE</code> previously left the driver thinking the branch was already prepared, so the follow-up <code>rollback(xid)</code> tried <code>ROLLBACK PREPARED</code> against a non-existent gid and returned <code>XAER_RMERR</code>. Transaction managers (Narayana) escalated this to <code>HeuristicMixedException</code>. With the fix, <code>rollback(xid)</code> takes the active-branch path and issues a plain <code>ROLLBACK</code>, which the server accepts cleanly. Fixes [Issue <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/3153">#3153</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/3153">pgjdbc/pgjdbc#3153</a>), [Issue <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/3123">#3123</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/3123">pgjdbc/pgjdbc#3123</a>). [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4114">#4114</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4114">pgjdbc/pgjdbc#4114</a>)</li> <li>fix: an updatable result set over an unqualified table name is now classified using only the table visible through <code>search_path</code>. When two schemas held a table with the same name and the same primary or unique index name but a different set of key columns, the driver took the union of both schemas' columns, so the result set could be wrongly rejected as not updatable [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4214">#4214</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4214">pgjdbc/pgjdbc#4214</a>). Supersedes [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/3400">#3400</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/3400">pgjdbc/pgjdbc#3400</a>).</li> <li>fix: <code>LargeObject.close()</code> now flushes a buffered output stream before marking the object closed, so closing a large object without an explicit <code>flush()</code> no longer drops buffered writes. The flush runs while the object is still open (it calls back into <code>LargeObject.write()</code>), and <code>lo_close</code> always runs afterward; a failure from <code>lo_close</code> no longer masks an earlier flush error, and the transaction is not committed when the flush failed [Issue <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4247">#4247</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4247">pgjdbc/pgjdbc#4247</a>) [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4248">#4248</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4248">pgjdbc/pgjdbc#4248</a>).</li> <li>fix: reject empty <code>timestamp</code>, <code>timestamptz</code>, and <code>date</code> text with a clear <code>SQLException</code> (SQLState <code>22007</code>) instead of an <code>ArrayIndexOutOfBoundsException</code> [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4278">#4278</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4278">pgjdbc/pgjdbc#4278</a>)</li> <li>fix: return null <code>CHAR_OCTET_LENGTH</code> for non-character columns [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4231">#4231</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4231">pgjdbc/pgjdbc#4231</a>)</li> <li>fix: honor scale in <code>ResultSet.getBigDecimal(int, int)</code> [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4211">#4211</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4211">pgjdbc/pgjdbc#4211</a>)</li> <li>fix: support <code>java.time</code> values in an updatable <code>ResultSet</code> <code>updateRow()</code> / <code>insertRow()</code> [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/3848">#3848</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/3848">pgjdbc/pgjdbc#3848</a>)</li> <li>fix: improve batching when the <code>RETURNING</code> clause contains <code>varchar</code> or <code>numeric</code> types [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4014">#4014</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4014">pgjdbc/pgjdbc#4014</a>)</li> <li>fix: correct <code>estimatedReceiveBufferBytes</code> accounting after a forced <code>Sync</code> [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4014">#4014</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4014">pgjdbc/pgjdbc#4014</a>)</li> <li>fix: avoid creating a transient <code>ResultSet</code> for describe-statement purposes, and restore the pre-describe path for generated-key batches [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4014">#4014</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4014">pgjdbc/pgjdbc#4014</a>)</li> <li>fix: add an explicit failure message when a multi-statement command executes in a batch [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4014">#4014</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4014">pgjdbc/pgjdbc#4014</a>)</li> <li>fix: detect <code>search_path</code> changes case-insensitively [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4216">#4216</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4216">pgjdbc/pgjdbc#4216</a>)</li> <li>fix: auto-detect the SSL key format instead of relying on the <code>.key</code> extension [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/3946">#3946</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/3946">pgjdbc/pgjdbc#3946</a>)</li> <li>fix: build PKIX trust anchors without a <code>KeyStore</code> so FIPS JVMs work [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4193">#4193</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4193">pgjdbc/pgjdbc#4193</a>)</li> <li>fix: use <code>gssResponseTimeout</code> rather than <code>sslResponseTimeout</code> for GSS connections [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4076">#4076</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4076">pgjdbc/pgjdbc#4076</a>)</li> <li>fix: skip the autosave savepoint for <code>SET LOCAL</code> / <code>SET SESSION TRANSACTION</code> [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4203">#4203</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4203">pgjdbc/pgjdbc#4203</a>)</li> <li>fix: do not throw <code>AssertionError</code> from <code>BatchResultHandler</code> on a closed connection [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4187">#4187</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4187">pgjdbc/pgjdbc#4187</a>)</li> <li>fix: reject <code>SQL_TSI_FRAC_SECOND</code> with an explicit, explained error [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4229">#4229</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4229">pgjdbc/pgjdbc#4229</a>)</li> <li>fix: reject a null URL in <code>Driver.acceptsURL</code> with a clear <code>NullPointerException</code> [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4205">#4205</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4205">pgjdbc/pgjdbc#4205</a>)</li> <li>fix: reject overlong inputs in <code>NumberParser.getFastLong</code> instead of silently wrapping [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4163">#4163</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4163">pgjdbc/pgjdbc#4163</a>)</li> <li>fix: reject out-of-range and NaN values in <code>PGInterval.setSeconds</code> [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4194">#4194</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4194">pgjdbc/pgjdbc#4194</a>)</li> <li>fix: close the socket when <code>PgConnection</code> setup fails after connect [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4161">#4161</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4161">pgjdbc/pgjdbc#4161</a>)</li> <li>fix: keep the <code>LazyCleanerImpl</code> cleanup task alive across a transient empty queue [PR <a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4038">#4038</a>](<a href="https://redirect.github.com/pgjdbc/pgjdbc/pull/4038">pgjdbc/pgjdbc#4038</a>)</li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/pgjdbc/pgjdbc/commit/3297557c6a8059d0d6e3522c79f0bd9a6f82ee07"><code>3297557</code></a> docs: add 42.7.13 release changelog (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4270">#4270</a>)</li> <li><a href="https://github.com/pgjdbc/pgjdbc/commit/d93d370984fbbccd099fc6466e4075ba46d8ec59"><code>d93d370</code></a> style: apply Autostyle to docs/ and .github/</li> <li><a href="https://github.com/pgjdbc/pgjdbc/commit/2e05ff9e3ea9e8249f25dcb7017984285f1f76b1"><code>2e05ff9</code></a> build: check docs/ and .github/ formatting with Autostyle</li> <li><a href="https://github.com/pgjdbc/pgjdbc/commit/b4a6087d2f07b578923a5272fa0155602ade9d40"><code>b4a6087</code></a> Adjust EditorConfig für Makefiles</li> <li><a href="https://github.com/pgjdbc/pgjdbc/commit/725cebbb4e13be777483bd916b19dfcd428b5f26"><code>725cebb</code></a> fix(jdbc): reject empty timestamp/timestamptz text with a clear error</li> <li><a href="https://github.com/pgjdbc/pgjdbc/commit/23a1b0dac5a8fc633cc163e883eb21f0219ea521"><code>23a1b0d</code></a> fix(scram): fail closed on channel-binding downgrade (no scram bump)</li> <li><a href="https://github.com/pgjdbc/pgjdbc/commit/0b4077a529b2448cc55a6ca87b2be8667243c9ab"><code>0b4077a</code></a> Bump pgjdbc version from 42.7.12 to 42.7.13 (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4269">#4269</a>)</li> <li><a href="https://github.com/pgjdbc/pgjdbc/commit/394800a38aebf54f9f293f6198e1fc5c8b19f10f"><code>394800a</code></a> fix: flush LargeObject output stream before marking closed (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4248">#4248</a>)</li> <li><a href="https://github.com/pgjdbc/pgjdbc/commit/83780f130e0c83a77bb67350603f3cca8d4b9bb2"><code>83780f1</code></a> Maintain consistency with the use of the word maintainer vs comitter (<a href="https://redirect.github.com/pgjdbc/pgjdbc/issues/4234">#4234</a>)</li> <li><a href="https://github.com/pgjdbc/pgjdbc/commit/d42cad5cd12a13197e68aa53f09a7721336dce7a"><code>d42cad5</code></a> fix(jdbc): classify updatable result set by search_path visibility</li> <li>Additional commits viewable in <a href="https://github.com/pgjdbc/pgjdbc/compare/REL42.7.11...REL42.7.13">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
ConnorYoh
|
8990f55e50 |
feat(storage): encryption at rest for stored files (per-team envelope encryption) (#7155)
# Description of Changes
PR1 of the encrypt-at-rest initiative: user files stored by Stirling (My
Files, workflow files) are now AES-256 encrypted at rest across all
three storage backends, with keys that never leave the deployment.
**What was changed**
- New `EncryptingStorageProvider` decorator wraps whichever
`StorageProvider` backend is configured (local / database / S3). It
encrypts on `store` (Tink AES-256-GCM streaming AEAD, 1 MiB segments)
and transparently decrypts on `load`; legacy plaintext blobs are
detected by magic sniff and pass through untouched, so mixed state is
safe and no migration is required to enable.
- Envelope-encryption key hierarchy: each blob gets a random per-file
DEK, wrapped by a per-team KEK stored (master-key-wrapped) in a new
`file_encryption_keys` registry table; the master key resolves like the
existing credential key — `stirling.security.fileEncryptionKey`
property, `STIRLING_FILE_ENCRYPTION_KEY` env var, or an auto-generated
owner-only `file-encryption.key` in the config dir (cluster mode
requires an explicit shared key, fail-fast).
- Self-describing blob format (`SPDFEAR1` header) carrying the key id,
plaintext length, and the wrapped DEK; the header prefix is bound as GCM
associated data to both the DEK wrap and the payload, so headers cannot
be transplanted between blobs.
- Enabled via `storage.encryption.enabled=true`, gated on a
Pro/Enterprise licence — **write side only**: decryption activates
whenever key rows exist, so switching the flag off or a lapsed licence
can never make previously encrypted files unreadable.
- Key status lifecycle (`ACTIVE`/`RETIRED`/`DISABLED`): `DISABLED` is a
reversible per-team kill switch that fails closed on read; no API path
deletes key material. A revoked download surfaces as **403 Forbidden**
("access revoked"), not a 500, since it is a deliberate policy state
rather than a fault.
- Startup self-check: a master key that cannot unwrap existing key rows
refuses to boot rather than silently starting a second key hierarchy.
- S3 presigned download URLs are suppressed for decorated storage (they
would serve ciphertext); the controller already falls back to
app-streamed downloads.
- `StoredFile`/`StoredObject` gain a nullable `encryption_key_id`
(ddl-auto, no migration); persisted sizes remain plaintext sizes so
quotas and UI are unchanged.
**Why**
Enterprise security questionnaires (and HIPAA/GDPR/CMMC buyers) require
encryption at rest with documented key management; files were previously
plaintext in every backend. Design doc and vendor/standards research
(Purview, Box KeySafe, Google CSE, ISO 32000-2) informed the approach.
## Manually tested end-to-end
Beyond the automated suite, the full flow was exercised against a
running backend (local provider, `storage.encryption.enabled=true`,
login enabled) via the storage API:
1. **Startup** — master key auto-generated with the "back this up"
warning; logs `master key initialised (AES-256-GCM, fingerprint …)` and
`Storage encryption at rest active (writes encrypted)`.
2. **Encrypted at rest** — uploaded a PDF containing a known marker
string; the blob on disk (371 B vs 219 B plaintext) began with the
`SPDFEAR1` header + key id + ciphertext, contained **no `%PDF` signature
and no marker** — not openable as a PDF straight off disk.
3. **Transparent access** — downloading the file through the API
returned it **byte-identical** to the original, marker intact; stored
`sizeBytes` stayed the plaintext size.
4. **Kill switch + reversibility** — set the team key's status directly
in the DB and restarted:
- `DISABLED` → download **failed closed** (`403`, "access to this
content is revoked"), zero plaintext served.
- `ACTIVE` again → file **fully recovered, byte-identical**. Disabling
is a reversible switch on a preserved key row, not destruction.
(The 403 mapping in step 4 was added in this PR after the manual run
first surfaced it as a generic 500.)
## Coming in later PRs
- **PR2 — ops & lifecycle:** audit events for
encrypt/decrypt/key-lifecycle; admin endpoints for the kill switch
(disable/enable) and key status; a background "encrypt existing files"
migration job for turning the feature on over pre-existing plaintext;
master-key rotation (re-wrap KEK rows). Also plans a
key-backup/fingerprint verification command.
- **PR3 — admin UI:** settings section (status, per-team key list with
disable/enable), encrypted-file badge in My Files, i18n.
- **Later:** per-**source** encryption for the Processor pipeline (the
`SOURCE` key scope is already reserved in the schema); pluggable
external KMS / BYOK master-key backends (Vault, AWS/Azure/GCP KMS);
optional FIPS-validated crypto module build for CMMC; and encrypted
egress (PDF-native AES-256) for files leaving the platform.
**Reviewer notes**
- New dependency: `com.google.crypto.tink:tink:1.23.0` (Apache-2.0, pure
Java — bundled in the boot jar, no Docker changes). Pulls protobuf-java
4.33.6, which clears the Aikido-flagged CVE-2024-7254. `./gradlew
checkLicense --no-parallel` passes.
- The `file-encryption.key` file is generated in the config dir on first
use and must be backed up; losing it makes encrypted files unrecoverable
(loud log warning + fingerprint exposed for backup verification).
- Tests cover round-trips on re-openable and one-shot (S3-style)
backends, multi-segment files, legacy passthrough, decrypt-only mode,
disabled-key fail-closed (now asserting the 403 mapping), header/payload
tamper rejection, key-creation races, and presigned-URL suppression.
---
## Checklist
### General
- [x] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [x] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [x] I have performed a self-review of my own code
- [x] My changes generate no new warnings
### Documentation
- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)
### Translations (if applicable)
- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)
### UI Changes (if applicable)
- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)
### Testing (if applicable)
- [x] I have run `task check` to verify linters, typechecks, and tests
pass
- [x] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
|
||
|
James Brunton
|
3bee6d212e |
Change pipelines to have 1 input and 1 output (#7121)
# Description of Changes Change pipelines so that sources and triggers are grouped into a list of inputs, so you can have a different trigger for each source in the list. This is necessary because triggers are not universally supported by all source types. If you wanted to have a pipeline pull from both a folder and an S3 bucket, the current system allows you to choose "Folder Watch" as the trigger, which will either do nothing or crash when it's paired with the S3 bucket. I've got reservations about actually allowing different triggers for every source because it allows for user workflows that I don't believe exist, like "I want this folder to be polled every minute and this other one to be polled every hour, but they should run the same tools and should output to the same place". Because of this (with agreement from Connor, Anthony and Matt) I've changed this PR to artificially limit pipelines to having 1 input & output at this stage. The backend is still shaped to support multiple inputs & outputs so it should be trivial to re-add support for them in the future if we decide we want to, but the UI can be much simpler and easier to understand with just 1 input and output. <img width="1262" height="521" alt="image" src="https://github.com/user-attachments/assets/809e6803-9f99-436d-9aeb-52dddf0906ff" /> |
||
|
ConnorYoh
|
b4a264239c |
fix(saas): provision a new user and their personal team atomically (#7193)
New SaaS accounts were landing with `team_id = null`. That state is unrecoverable: portal access derives from leading a team, and signup is the only place one is assigned. Five things had to be fixed, all on the signup path. Only the last is a behaviour change you'd notice. ### 1. Shared-PK entity was routed to `merge()` `SaasUserExtensions` pre-sets its `@MapsId` id in the constructor, so Spring Data's id-nullness check treated a brand-new row as existing and `save()` failed with `AssertionFailure: null identifier`. Now implements `Persistable` and decides on the creation timestamp — the idiom already used by `ProcessedFileEntity` and `SourceDocCountEntity`. This was the blocker. It threw on every signup, and because the failure was swallowed (see 3) every new account was stranded. ### 2. User and team were committed separately `createUser()` is annotated `@Transactional` but is called as `this.createUser(...)`, and self-invocation bypasses the proxy — so the annotation did nothing. `saveUser()` and `ensurePersonalTeam()` each committed in their own transaction, leaving a window where a **committed user was visible with `team_id = null`**. Parallel requests entering that window each provisioned a team, producing duplicates (observed: teams 160/161 and 162/163 for one user). Both writes now happen in one transaction via `SaasTeamService.saveUserWithPersonalTeam()`. The window is gone, so there is nothing left to race over. ### 3. A failed team create was swallowed The old code logged at WARN and committed the user anyway. It now propagates: the shared transaction rolls the user back, the request 401s, and a retry starts clean. Nothing half-built is committed. This is the deliberate trade — a transient failure now surfaces instead of silently producing an account that can never reach the portal. ### 4. Per-request healing removed `recoverMissingTeam` (added in #7180) ran on **every authenticated request** whose user had no team, with no mutual exclusion. Under a burst of parallel requests it was itself a source of concurrent provisioning. Provisioning belongs to signup alone. ### 5. Policy seeding could not run `@TransactionalEventListener(AFTER_COMMIT)` leaves the *completed* transaction bound to the thread, so `JpaPolicyStore.save`'s `@Transactional` joined it instead of opening a live one — and its `FOR UPDATE` lock threw `TransactionRequiredException`. Now seeded in `BEFORE_COMMIT`: the lock has a live transaction, rollback safety is unchanged (a rolled-back team still leaves no policy), and it stays on a single pooled connection. ## Verified `:saas:test` green, both spotless gates green, on top of current `main`. Manually on a live signup: **one** team per user, and the concurrent-signup race resolves correctly through the pre-existing unique-constraint catch (`users_supabase_auth_id_key` violation → refetch the winner). 12 filter tests needed updating. Two of them asserted behaviour this PR deliberately removes (`personalTeamFailureSwallowed`, `assignsTeamWhenMissing`), so they were rewritten to assert the new contract rather than re-stubbed into passing. ## Not in scope - **Existing stranded accounts** are not repaired — with the healer gone, nothing fixes them on the request path. They need a one-off backfill or deletion. - **A DB-level invariant.** A partial unique index (`UNIQUE(created_by_user_id) WHERE is_personal`) would make duplicate personal teams impossible rather than merely unreachable. Wanted, but it is a Supabase migration in the SaaS repo, so it is deliberately separate. - **Per-request auth cost.** The filter still does two remote-Postgres round-trips per authenticated request; a frontend request storm makes that expensive. Being handled separately. |
||
|
James Brunton
|
999b5e5995 |
Add persistent outputs to Processor (#7071)
# Description of Changes <img width="1270" height="487" alt="image" src="https://github.com/user-attachments/assets/64894e2b-aab9-42ab-96c2-2c11ba427b52" /> Change policies to point towards a source for its output instead of a dynamically defined output location for the pipeline. This allows for easy reuse of outputs in different pipelines and makes it impossible to break complex pipelines by accidentally updating the source but not the output and vice versa. Also makes outputs a list to match the inputs, so it's possible for a pipeline to output to multiple locations. We should consider whether we want to continue calling these Sources since they're now being used as both inputs and outputs, but that decision is beyond the scope of this PR. Also updates the existing S3 DB migration script and adds a new one to migrate to the new schema. Neither of these scripts are possible with SQL since it involves parsing and restructuring JSON. I've updated them so that they only ever run once on startup and mark themselves as completed. |
||
|
ConnorYoh
|
66b80a80c0 |
fix(saas): personal teams must be allowed to share a name (new signups get no team) (#7180)
## The bug
Every SaaS signup after the very first one is created with `team_id =
NULL`, no team membership and no `home_team_id`. A brand-new account:
```
user_id | username | team_id | authenticationtype | home_team_id | memberships
952 | hedewot627@candaba.com | null | web | null | null
```
Since #7070 derives Processor access from leading a team, these accounts
are silently redirected out of the Processor and back to the editor.
## Cause
`SaasTeamService.createPersonalTeam` names every personal team the
literal `"My Team"`, and `stirling_pdf.teams.name` was unique — so the
insert throws a duplicate-key error for the second account onwards. Team
creation is best-effort (caught, logged at WARN), so the account is
created anyway, permanently team-less.
Migration `20251211000000` had already dropped that constraint for
exactly this reason, but it dropped it **by name** while the entity
still declared `@Column(unique = true)`. With Flyway retired for `:saas`
(#7100), `ddl-auto=update` reconciles the schema — so Hibernate
re-created the constraint on the next boot under a generated name the
old `DROP` could never match.
The data bug predates #7070; that PR only made it visible.
## Changes
- **`Team.name` no longer unique.** `TeamController` already enforces
uniqueness for admin-created teams (`existsByNameIgnoreCase` on create
and rename, 409), so nothing user-facing changes. `findByName` is only
used for the `Default`/`Internal` system teams.
- **Existing team-less accounts recover on authentication.** Signup is
the only other place a team is assigned and nothing back-fills
`team_id`, so without this they stay locked out. Guests excluded by
design; healthy accounts short-circuit on a null check (`team` is
`EAGER`).
- **Tests:** team recovered, existing team untouched, guest stays
team-less.
## Deploy order
Needs `20260806000000_teams_name_drop_unique` (SaaS repo, `v3`) to drop
the constraint from the live schema — **deployed after this**, or
Hibernate re-adds it on the next boot.
## Verification
`:saas:compileJava`, `:proprietary:compileJava`, spotless on both, and
the `:saas` team/auth-filter tests (`TeamRecovery`: 3 tests, 0
failures).
|
||
|
James Brunton
|
29002d0b82 |
Improve UX around source folders in Processor (#7101)
# Description of Changes Adds implicitly defined folders to the list of locations that folder sources can look in, including the legacy watchedFolder folders, and the server storage location (if enabled). Also adds a settings UI for defining the list of allowed folders instead of having to manually edit `settings.yml` (please excuse the styling, that's the standard styling of the Processor, hoping it gets fixed by one of the styling PRs). <img width="888" height="786" alt="image" src="https://github.com/user-attachments/assets/cf6d0705-adcf-463c-8e80-6901a652068b" /> <img width="1103" height="713" alt="image" src="https://github.com/user-attachments/assets/b7244592-249a-4149-994f-3a2b750f25f9" /> |
||
|
Anthony Stirling
|
1e2895a79f |
Add external-API integrations plus pipeline steps (#7098)
New Generic API mode with examples and integrations setup around it - Adds an integration operations catalogue so external-API connections (e.g. Microsoft Purview) can be used as policy pipeline steps - New generic external-API step calls a configured connection during a policy run, with a verdict gate to pass/fail documents on the response - Purview sensitivity-labelling step applies labels to processed documents, gated behind the Purview connection being configured (WIP to be changed later) --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
Anthony Stirling
|
8de94ff152 |
Ai customization settings (#7069)
# Description of Changes AI settings customisation in settings menu, as part of this also tested and fixed ollama and other 3rd party AI integrations - Adds an admin AI settings UI for customizing AI behaviour, including per-provider model and API-key configuration - Backend pushes AI config changes to the Python engine at runtime via a config-push bridge, so changes apply without a restart - Config-push is gated off in SaaS; engine now drains background tasks on shutdown instead of cancelling them --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
Anthony Stirling
|
b3875d3149 |
Add heuristic classification (#7050)
# Description of Changes - Adds a non-AI heuristic classification engine that classifies documents client-side in the browser when AI is disabled - Classification is billed as a policy run via a fast, non-blocking meter endpoint; a default Classification policy is seeded per team - Enables the policy engine by default --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
Anthony StirlingandEthanHealy01
|
357eb77f94 |
Portal: multiple named personal API keys with per-key usage tracking (#6961)
# Description of Changes Multiple named **personal** API keys per user, replacing the single opaque per-user key. - Create (name + one-time secret), list, and revoke named keys from the portal Infrastructure → API Keys tab. Works self-hosted and SaaS (`X-API-KEY`). - Per-key usage stats (today / trailing 30 days / lifetime); API-processed documents are attributed to the specific key in the processor's Documents feed. - The legacy single per-user key keeps working and is lazily represented as a named key. Rotating it revokes its migrated shadow row so the old secret stops authenticating. - Per-user (not per-key) rate limiting plus a per-user active-key cap, so minting keys can't multiply the daily quota. Name-length cap; race-safe migration and usage recording. Keys are strictly personal: one owner, full access, no sharing. Team-shared / scoped keys and per-key access levels were intentionally left out of this PR to keep it small and easy to review; they can follow as a separate, focused change. > Note: the screenshots from the original revision showed an earlier team-scoped design and need refreshing. --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [x] I have performed a self-review of my own code - [x] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [x] I have run `task check` to verify linters, typechecks, and tests pass - [x] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. --------- Co-authored-by: EthanHealy01 <80844253+EthanHealy01@users.noreply.github.com> |
||
|
Anthony Stirling
|
3bf0019d7c |
Webhook policy source (#7051)
# Description of Changes Create custom webhooks as a source, allows file pushes toa custom made endpoint with custom auth ID - Adds webhook as a policy source: external systems push documents to a receiver endpoint, which stages the files locally and triggers the policy run - Requests are authenticated with HMAC signatures; receiver hardened with bounded body reads and server-minted IDs - Uses the same team-scoped IntegrationConfig connection model as the S3 source, with matching portal UI (source type, icon, wizard) - Includes a policies-gated Cucumber feature covering the receiver end-to-end --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
brios
|
e24a30828b |
refactor(api): replace deprecated APIs with their modern equivalents (#6434)
# Description of Changes This PR resolves deprecation warnings and addresses compiler errors resulting from the transition to Spring Security 7.x., as well Jackson 3 and general Java. * Replaced all usages of `asText()`/`isTextual()` with `asString()`/`isString()` in JSON parsing logic across `FormPayloadParser.java`, `ApiEndpoint.java`, and `KeygenLicenseVerifier.java` to ensure consistent and type-safe string * Updated `CustomSaml2AuthenticatedPrincipal` to implement `Saml2ResponseAssertionAccessor`, added a `responseValue` field, and provided additional getter methods and type-safe attribute accessors. * Switched from constructing `URL` objects directly from strings to using `URI.create(...).toURL()` in `UIDataTessdataController.java` for improved URL safety and parsing. <!-- Please provide a summary of the changes, including: - What was changed - Why the change was made - Any challenges encountered Closes #(issue_number) --> --- ## Checklist ### General - [x] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [x] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [x] I have performed a self-review of my own code - [x] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [x] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
briosandAnthony Stirling
|
e2ea720fc8 |
refactor(api): replace regex literals with compiled patterns for improved performance and readability (#6511)
# Description of Changes This pull request refactors several utility classes and controllers to replace inline regular expression usage with precompiled `Pattern` constants. This change improves performance, consistency, and maintainability by ensuring that regex patterns are compiled only once and reused throughout the codebase. Additionally, it enhances code clarity and security in filename and SQL content sanitization. <!-- Please provide a summary of the changes, including: - What was changed - Why the change was made - Any challenges encountered Closes #(issue_number) --> --- ## Checklist ### General - [X] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [X] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [X] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [X] I have performed a self-review of my own code - [X] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [X] I have run `task check` to verify linters, typechecks, and tests pass - [X] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. --------- Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com> |
||
|
James Brunton
|
5e3e89ccb2 |
Fix existing teams logic (#7070)
# Description of Changes Paired with https://github.com/Stirling-Tools/Stirling-PDF-SaaS/pull/320 Fixes the following bugs we found when testing the SaaS release: - Existing users couldn't join teams - this was because they were the last leader of their team, so it'd be left orphaned). Users now have a 'home team', which can have no members if they join another team, but they can then go back to it later. - Existing leaders didn't have unlimited seats - `saas_teams_extensions` had no row for them, so the app fell back to `max_seats=1`. The migration script fixes it. - Members without Processor access could still access the Processor - It was just checking "Are you the leader of **any** team", instead of the user's active team. |
||
|
Anthony Stirling
|
79dc7d5615 |
Multi node cluster fixes (#7025)
- Exclude DataRedisRepositoriesAutoConfiguration (cluster crash-loop fix) - Share JWT signing keys via the DB + require a shared credential key in cluster mode - Make policy run status/listing visible across nodes --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
James BruntonandEthanHealy01
|
0fc2958daa |
Add explicit length to member column to avoid ddl resizing (#7109)
# Description of Changes There's currently a column size inconsistency between the SaaS v3 DB and the main Java code which causes the backend to fail to start up when connected to a fresh DB. This is because the column previously was width 255, but now it's officially width 50, but the Java type is still implicitly `varchar(255)` because there's no length attribute. If it's a fresh DB, Postgres throws an error that it can't expand the column (this doesn't error on an existing DB because the column is already wide enough behind the scenes). Co-authored-by: EthanHealy01 <80844253+EthanHealy01@users.noreply.github.com> |
||
|
LudyandCopilot
|
bb830e5711 |
build: upgrade google-java-format and restore strict Spotless validation (#7091)
# Description of Changes - Upgraded google-java-format from 1.28.0 to 1.35.0. - Removed the broad `suppressLintsFor` workaround for the `google-java-format` step. - Ensured the shared `gradle/spotless.gradle` configuration is recognized by the relevant CI path filters and repository automation. - Kept the shared formatter configuration available to all backend modules. - Verified that google-java-format 1.35.0 runs successfully on JDK 25 for the Common, Core, and SaaS modules. - Confirmed that the previous claim about a general Guava 32.x crash on JDK 24/25 no longer justifies suppressing all formatter lint failures. ### Verification Verified with Temurin JDK 25.0.3 and google-java-format 1.35.0. The formatter still depends on Guava 32.1.3-jre, and no `suppressLintsFor` configuration is present. ```bash ./gradlew \ :common:spotlessJavaCheck \ :stirling-pdf:spotlessJavaCheck \ --rerun-tasks ``` Result: ```text > Task :common:spotlessJava > Task :common:spotlessJavaCheck > Task :stirling-pdf:spotlessJava > Task :stirling-pdf:spotlessJavaCheck BUILD SUCCESSFUL in 26s 4 actionable tasks: 4 executed ``` Using `--rerun-tasks` ensured that the formatter was executed and that the result did not come from the Gradle task cache. --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |
||
|
Ludy
|
c6e84a2124 |
fix(admin-settings): correctly mask Telegram bot tokens in settings output (#6822)
# Description of Changes
- What was changed
- Fixed the sensitive-field detection logic in `AdminSettingsController`
so `botToken` is matched correctly after lowercasing the field name.
- This ensures Telegram bot tokens are masked consistently in admin
settings responses.
- Why the change was made
- The previous check used `lowerField.contains("botToken")`, which could
never match after converting the field name to lowercase.
- As a result, `botToken` values could remain visible in masked settings
output.
---
## Checklist
### General
- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings
### Documentation
- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)
### Translations (if applicable)
- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)
### UI Changes (if applicable)
- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)
### Testing (if applicable)
- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
|
||
|
Anthony Stirling
|
939afef14f |
perf(portal): collapse admin-roster N+1 + session indexes (#7008)
# Description of Changes Collapses the portal admin-roster endpoint (`getAdminSettingsData`, `/api/v1/proprietary/ui-data/admin-settings`) from a per-user N+1 into a constant set of queries, and adds the missing session/user/team-membership indexes. **Verified on H2 and real Postgres 16, 2,000-user roster:** 10,601 → 7 SQL statements, 600 → 0 writes-during-a-GET, O(N) → O(1). Portal-access resolution is proven equivalent to the per-user check (parity test), and a scaling guard fails the build if the endpoint ever regresses. Also in scope (same controller / session subsystem): `getLoginData` counts instead of loading the whole user table; `getTeamDetailsData` fetch-joins authorities; `SessionScheduled` uses one bulk expire + a bounded purge. Behaviour note: the roster "active" flag now reflects *any* live session (a strict superset of the old "newest session only") — no user who was active is ever shown inactive. --- ## Checklist ### General - [x] I have performed a self-review of my own code - [x] My changes generate no new warnings ### Testing - [x] I have run backend `task check` (spotless + full backend test suite) — all green - [x] I have tested my changes locally (before/after benchmark on H2 + Postgres) |
||
|
James Brunton
|
ed58d90ab8 |
Remove policies feature flag (#7031)
# Description of Changes Removes the feature flags for enabling policies on both the backend and frontend. We shouldn't be releasing another self-hosted release that doesn't include policies, so it makes sense to do this now. Builds that don't have the Processor will just not run policies because they won't have any. Beyond that, the API should always be available, but checks whether the user actually has the entitlements to run policies (whether they have credits/a payment method available) |
||
|
James Brunton
|
4ee54243f3 |
Remove encryption from stored policies JSON (#7035)
# Description of Changes The policies stored in the DB are currently encrypted at rest, because one version in the past included S3 keys. These are now stored properly in the credentials system and I've manually removed the only policy that used S3 (it was very recently released). Since there's no S3 (or other) credentials in the policies stored JSON now, we might as well just decrypt them. This PR pairs with #7034 to fix the issues - #7034 makes it resilient to crashing when attempting to load encrypted JSON that's been encrypted with the wrong key, and this makes it so if it does load any encrypted policies, they'll be re-saved decrypted, so we should have a vanishingly small number of encrypted policies over time. |
||
|
Anthony Stirling
|
663bb32b2c |
Skip unreadable policy source/policy rows instead of crashing :) (#7034)
# Description of Changes Thanks james for the prod issue :) --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
EthanHealy01
|
7f01bcdc44 |
Classifier setup as a processor policy (#7012)
## Overview Adds a **Classification policy** to the processor's policy catalogue, set up the same way as the Security policy. This moves classifier configuration out of the editor (where the labels UI landed in #6898 and was then removed with the rest of the editor's policy-management surface in #6932) and into the processor, which is now the single place policies are configured. ## What it does - **Classification card** in the processor policy catalogue. Always shown, but **setup is locked until the backend reports the AI engine is on** — so admins can see the capability they're missing rather than it being hidden entirely. - **Setup wizard** mirrors Security: the workflow step shows the team's **classification label editor** (reused `LabelsEditor`/`LabelsEditorModal` — add box, chip grid, per-label icon picker, import/export, reset) instead of tool toggles, since classify is a single non-configurable step. - On enable, the team's label vocabulary is **seeded with the 268 built-in defaults** (clobber-safe: only when the team has none). On upload the document is classified against the team's labels and tagged; on SaaS with the engine on, files group by category in the editor sidebar. ## Reuse & consolidation - Reuses the existing labels table, `labelsFile` helpers, and default vocabulary. Labels read/write through the processor's own `apiClient.local` (not the editor's axios client) so auth/base routing stays explicit; the wire shape is shared. - Consolidates policy-category icons into a shared, **id-keyed** `policyCategoryIcon` util (outline glyphs) used by both the editor and the processor, replacing the processor's emoji-glyph map (and the stray `schedule` key that rendered a bare dot). ## Testing - `task frontend:typecheck:{core,proprietary,portal}`, `frontend:lint:eslint`, `frontend:test` (156 files / 1305 tests) — all green. - Verified in Storybook: the Classification card renders, the setup wizard shows the label editor (268 defaults), and the full labels editor opens with icons/import/export/reset. Added an MSW handler for the app-config + labels endpoints and a `Classification` wizard story. ## Notes for reviewers - The AI-engine gate reads the public `/api/v1/config/app-config`; classification labels use `/api/v1/classification/labels` (team-scoped, team-lead/admin-gated, `policies.enabled`); the classify step hits `/api/v1/ai/tools/classify-and-label` — all pre-existing backend from #6898. - Known parity behavior (matches the editor hook): a transient failure loading team labels falls back to showing the defaults; not changed here to avoid diverging the two hooks. |