mirror of
https://github.com/Stirling-Tools/Stirling-PDF.git
synced 2026-09-03 05:10:16 +03:00
82e1bd62a2ad166312656b657f35978dbe59970a
232
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Ludy
|
b10fc1b2de |
fix(java): prevent executor, task, regex, and stream resource leaks (#7284)
# Description of Changes - Added graceful shutdown handling for service-owned executors in `JobExecutorService`, `PolicyEngine`, and `AsyncConfig`. - Added expiration and cleanup for abandoned pending jobs in `TaskManager`. - Replaced the unbounded regex pattern cache with a bounded cache limited to 512 entries. - Ensured `Files.walk()` is closed correctly in `MobileScannerService`. - These changes prevent unbounded heap growth, lingering virtual-thread executors, and file-descriptor leaks. - Added configurable pending-job expiration through `stirling.job.pendingExpiryMinutes`, defaulting to 24 hours. --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
brios
|
e560ee4cc4 |
chore(deps): update junrar dependency to version 8.0.0 (#7210)
# Description of Changes Since this was a major version update i tested manually, afterwards figured i'll submit as PR. This version of junrar adds long-awaited (by me) RAR 5 support to the library. RAR 5 is newest version of the RAR file format and was not available in previous Junrar version, but is somewhat common for CBR files to be RAR 5. For junrar release notes see: https://github.com/junrar/junrar/releases/tag/v8.0.0 Changes: - Bumped junrar dep to version 8.0.0 <!-- Please provide a summary of the changes, including: - What was changed - Why the change was made - Any challenges encountered Closes #(issue_number) --> --- ## Checklist ### General - [X] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [X] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [X] I have performed a self-review of my own code - [X] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [X] I have run `task check` to verify linters, typechecks, and tests pass - [X] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
brios
|
a2dd0298dc |
refactor(api): replace length checks with isEmpty (#7214)
# Description of Changes Stylistic problem reported by static analyzer. Changes: * Replaced `sb.length() > 0` and `sb.length() == 0` with `!sb.isEmpty()` and `sb.isEmpty()` for `StringBuilder`, `String`, and collections throughout the codebase, improving readability and aligning with modern Java best practices. <!-- Please provide a summary of the changes, including: - What was changed - Why the change was made - Any challenges encountered Closes #(issue_number) --> --- ## Checklist ### General - [x] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [x] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [x] I have performed a self-review of my own code - [x] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [x] I have run `task check` to verify linters, typechecks, and tests pass - [x] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
Anthony Stirling
|
921bdac4b7 |
Desktop installer fixes (#7174)
# Description of Changes
* Adds a `windows-11-arm` CI/release leg (NSIS, Microsoft JDK 25,
updater keys); JPDFium natives deliberately excluded
(`jpdfiumPlatforms=none`) until published, so don't ship ARM64
installers to users yet
* Defaults `WEBKIT_DISABLE_DMABUF_RENDERER=1` on Linux (crash switching
tools on NVIDIA)
* Strips the bundled libwayland from AppImages (blank window on Fedora
Wayland)
* Blocks off-app webview navigation + window drop guard + close failsafe
(drag-drop bricks the app)
* 120s startup grace before the backend is declared unhealthy, restart
success only announced after a real health check ("Backend stopped
unexpectedly" spam and likely the OAuth port churn)
* Verified: green `windows-arm64` build (234 MB NSIS artifact) and green
Linux run with libwayland confirmed stripped
* JPDFium fixes for multi threading issues
---
## Checklist
### General
- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings
### Documentation
- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)
### Translations (if applicable)
- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)
### UI Changes (if applicable)
- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)
### Testing (if applicable)
- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
|
||
|
Anthony Stirling
|
21dff695fe |
Add SFTP, FTP and SMB network sources to the processor (#7153)
# Description of Changes Add SFTP, FTP and SMB network sources to the processor plus UI change to enable it --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
James BruntonandAnthony Stirling
|
cd199c8659 |
Define tool inputs & outputs in a structured way (#7204)
# Description of Changes Change tool APIs to use structured definitions for input/output/type info because we need that info to be able to validate whether policies can actually successfully work based on whether one tool accepts the output of another. There were various bugs in the previous string definitions because of either misspellings or just incorrect definitions, so I've gone through and fixed all that I can find. <img width="729" height="271" alt="image" src="https://github.com/user-attachments/assets/08357e96-6fbb-4b9c-ba4d-8995420c7b86" /> <img width="749" height="264" alt="image" src="https://github.com/user-attachments/assets/76f46284-1866-4b64-b1ed-2480e01866e9" /> <img width="402" height="636" alt="image" src="https://github.com/user-attachments/assets/8f7a36ca-2845-4f14-a2df-ec9c772e66f6" /> <img width="393" height="317" alt="image" src="https://github.com/user-attachments/assets/46d8b891-9820-4ce3-8109-a8b782277037" /> --------- Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com> |
||
|
ConnorYoh
|
8990f55e50 |
feat(storage): encryption at rest for stored files (per-team envelope encryption) (#7155)
# Description of Changes
PR1 of the encrypt-at-rest initiative: user files stored by Stirling (My
Files, workflow files) are now AES-256 encrypted at rest across all
three storage backends, with keys that never leave the deployment.
**What was changed**
- New `EncryptingStorageProvider` decorator wraps whichever
`StorageProvider` backend is configured (local / database / S3). It
encrypts on `store` (Tink AES-256-GCM streaming AEAD, 1 MiB segments)
and transparently decrypts on `load`; legacy plaintext blobs are
detected by magic sniff and pass through untouched, so mixed state is
safe and no migration is required to enable.
- Envelope-encryption key hierarchy: each blob gets a random per-file
DEK, wrapped by a per-team KEK stored (master-key-wrapped) in a new
`file_encryption_keys` registry table; the master key resolves like the
existing credential key — `stirling.security.fileEncryptionKey`
property, `STIRLING_FILE_ENCRYPTION_KEY` env var, or an auto-generated
owner-only `file-encryption.key` in the config dir (cluster mode
requires an explicit shared key, fail-fast).
- Self-describing blob format (`SPDFEAR1` header) carrying the key id,
plaintext length, and the wrapped DEK; the header prefix is bound as GCM
associated data to both the DEK wrap and the payload, so headers cannot
be transplanted between blobs.
- Enabled via `storage.encryption.enabled=true`, gated on a
Pro/Enterprise licence — **write side only**: decryption activates
whenever key rows exist, so switching the flag off or a lapsed licence
can never make previously encrypted files unreadable.
- Key status lifecycle (`ACTIVE`/`RETIRED`/`DISABLED`): `DISABLED` is a
reversible per-team kill switch that fails closed on read; no API path
deletes key material. A revoked download surfaces as **403 Forbidden**
("access revoked"), not a 500, since it is a deliberate policy state
rather than a fault.
- Startup self-check: a master key that cannot unwrap existing key rows
refuses to boot rather than silently starting a second key hierarchy.
- S3 presigned download URLs are suppressed for decorated storage (they
would serve ciphertext); the controller already falls back to
app-streamed downloads.
- `StoredFile`/`StoredObject` gain a nullable `encryption_key_id`
(ddl-auto, no migration); persisted sizes remain plaintext sizes so
quotas and UI are unchanged.
**Why**
Enterprise security questionnaires (and HIPAA/GDPR/CMMC buyers) require
encryption at rest with documented key management; files were previously
plaintext in every backend. Design doc and vendor/standards research
(Purview, Box KeySafe, Google CSE, ISO 32000-2) informed the approach.
## Manually tested end-to-end
Beyond the automated suite, the full flow was exercised against a
running backend (local provider, `storage.encryption.enabled=true`,
login enabled) via the storage API:
1. **Startup** — master key auto-generated with the "back this up"
warning; logs `master key initialised (AES-256-GCM, fingerprint …)` and
`Storage encryption at rest active (writes encrypted)`.
2. **Encrypted at rest** — uploaded a PDF containing a known marker
string; the blob on disk (371 B vs 219 B plaintext) began with the
`SPDFEAR1` header + key id + ciphertext, contained **no `%PDF` signature
and no marker** — not openable as a PDF straight off disk.
3. **Transparent access** — downloading the file through the API
returned it **byte-identical** to the original, marker intact; stored
`sizeBytes` stayed the plaintext size.
4. **Kill switch + reversibility** — set the team key's status directly
in the DB and restarted:
- `DISABLED` → download **failed closed** (`403`, "access to this
content is revoked"), zero plaintext served.
- `ACTIVE` again → file **fully recovered, byte-identical**. Disabling
is a reversible switch on a preserved key row, not destruction.
(The 403 mapping in step 4 was added in this PR after the manual run
first surfaced it as a generic 500.)
## Coming in later PRs
- **PR2 — ops & lifecycle:** audit events for
encrypt/decrypt/key-lifecycle; admin endpoints for the kill switch
(disable/enable) and key status; a background "encrypt existing files"
migration job for turning the feature on over pre-existing plaintext;
master-key rotation (re-wrap KEK rows). Also plans a
key-backup/fingerprint verification command.
- **PR3 — admin UI:** settings section (status, per-team key list with
disable/enable), encrypted-file badge in My Files, i18n.
- **Later:** per-**source** encryption for the Processor pipeline (the
`SOURCE` key scope is already reserved in the schema); pluggable
external KMS / BYOK master-key backends (Vault, AWS/Azure/GCP KMS);
optional FIPS-validated crypto module build for CMMC; and encrypted
egress (PDF-native AES-256) for files leaving the platform.
**Reviewer notes**
- New dependency: `com.google.crypto.tink:tink:1.23.0` (Apache-2.0, pure
Java — bundled in the boot jar, no Docker changes). Pulls protobuf-java
4.33.6, which clears the Aikido-flagged CVE-2024-7254. `./gradlew
checkLicense --no-parallel` passes.
- The `file-encryption.key` file is generated in the config dir on first
use and must be backed up; losing it makes encrypted files unrecoverable
(loud log warning + fingerprint exposed for backup verification).
- Tests cover round-trips on re-openable and one-shot (S3-style)
backends, multi-segment files, legacy passthrough, decrypt-only mode,
disabled-key fail-closed (now asserting the 403 mapping), header/payload
tamper rejection, key-creation races, and presigned-URL suppression.
---
## Checklist
### General
- [x] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [x] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [x] I have performed a self-review of my own code
- [x] My changes generate no new warnings
### Documentation
- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)
### Translations (if applicable)
- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)
### UI Changes (if applicable)
- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)
### Testing (if applicable)
- [x] I have run `task check` to verify linters, typechecks, and tests
pass
- [x] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
|
||
|
ConnorYoh
|
46865cd154 |
Add Auto Rotate tool that detects and fixes page orientation (#7152)
## What
New **Auto Rotate** tool: give it any PDF and it detects each page's
correct orientation and sets `/Rotate` so every page displays upright.
Lossless — only the page rotation metadata changes, content is never
re-rendered.
New endpoint: `POST /api/v1/misc/auto-rotate-pdf`, plus an editor tool
registered next to Rotate.
## How it works
Two-tier detection, per page, both expressed as an additive clockwise
`/Rotate` correction:
1. **Embedded-text fast path** (`AutoRotateDetection`): dominant glyph
direction via `PDFTextStripper`/`TextPosition.getDir()`. Trusted only
with >= 30 glyphs at >= 95% agreement. Near-instant for born-digital
PDFs and needs no external tools. Correction = `(glyphDir -
pageRotation) mod 360` — the sign conventions are pinned by
parameterized fixture tests covering all text-angle x `/Rotate`
combinations.
2. **Tesseract OSD fallback**: pages the text path can't decide are
rendered at 300 DPI grayscale and run through `tesseract --psm 0`.
Corrections apply only above a confidence threshold (default 14.0,
matching OCRmyPDF's `--rotate-pages-threshold`). Rendering honours the
existing `/Rotate`, so the verdict is always additive.
**Conservative by default**: blank pages, mixed-direction pages, and
low-confidence verdicts are skipped, never guessed — the failure mode to
avoid is making a correct page wrong.
### API surface
- `detectionMode`: `auto` (default) | `text` | `osd` — forcing one
method is useful for testing
- `confidenceThreshold`: minimum OSD confidence to apply a correction
- `dryRun=true`: returns a JSON per-page report instead of the PDF
- `pageRotations={"1":90,...}`: applies precomputed corrections without
detection
The frontend uses analyze-then-apply (dryRun, then pageRotations) so
detection runs exactly once per file, and the analysis report can be
shown in the UI.
### UI
The tool's results panel shows a **detection report** for
debugging/tuning: per page — method badge (Text / OCR / Skipped),
confidence score (glyph-dominance % for text, raw OSD score for OCR),
applied rotation, and a skip reason (too little text, mixed directions,
below threshold, OCR not installed...). Settings expose detection mode
and the OSD threshold.
### Dependency handling
Registered in `PageOps` only — deliberately **not** gated on the
`tesseract` group, because the text path works without Tesseract. The
controller checks `isGroupEnabled("tesseract")` at runtime; when it's
missing, scanned pages are skipped with a visible `tesseractUnavailable`
note instead of the whole tool disappearing.
## Testing
- 14 detection unit tests: all text-angle x `/Rotate` fixture
combinations (pins the direction conventions), dominance/glyph-count
guards, OSD output parsing
- 6 controller tests: dryRun report, correction application, explicit
pageRotations, tesseract-unavailable reporting, input validation
- Frontend: typecheck (core/desktop/proprietary), ESLint, Prettier, all
i18n audit tests
- **Live, text path**: fixture with pages at `/Rotate` 0/90/180/270 ->
all pages return upright; report UI verified in the browser
- **Live, OSD path**: image-only "scan" fixture (no text layer) with
pages upright/180/90 -> all detected by OSD at conf ~15-17 and
corrected; closed-loop re-analysis of the output reports 0 pages to
rotate with *higher* confidence than the input
Out of scope: skew correction (that's the OCR tool's `--deskew`); this
fixes 90-degree-multiple orientation only.
|
||
|
James Brunton
|
29002d0b82 |
Improve UX around source folders in Processor (#7101)
# Description of Changes Adds implicitly defined folders to the list of locations that folder sources can look in, including the legacy watchedFolder folders, and the server storage location (if enabled). Also adds a settings UI for defining the list of allowed folders instead of having to manually edit `settings.yml` (please excuse the styling, that's the standard styling of the Processor, hoping it gets fixed by one of the styling PRs). <img width="888" height="786" alt="image" src="https://github.com/user-attachments/assets/cf6d0705-adcf-463c-8e80-6901a652068b" /> <img width="1103" height="713" alt="image" src="https://github.com/user-attachments/assets/b7244592-249a-4149-994f-3a2b750f25f9" /> |
||
|
Anthony Stirling
|
1e2895a79f |
Add external-API integrations plus pipeline steps (#7098)
New Generic API mode with examples and integrations setup around it - Adds an integration operations catalogue so external-API connections (e.g. Microsoft Purview) can be used as policy pipeline steps - New generic external-API step calls a configured connection during a policy run, with a verdict gate to pass/fail documents on the response - Purview sensitivity-labelling step applies labels to processed documents, gated behind the Purview connection being configured (WIP to be changed later) --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
Anthony Stirling
|
8de94ff152 |
Ai customization settings (#7069)
# Description of Changes AI settings customisation in settings menu, as part of this also tested and fixed ollama and other 3rd party AI integrations - Adds an admin AI settings UI for customizing AI behaviour, including per-provider model and API-key configuration - Backend pushes AI config changes to the Python engine at runtime via a config-push bridge, so changes apply without a restart - Config-push is gated off in SaaS; engine now drains background tasks on shutdown instead of cancelling them --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
Anthony Stirling
|
3bf0019d7c |
Webhook policy source (#7051)
# Description of Changes Create custom webhooks as a source, allows file pushes toa custom made endpoint with custom auth ID - Adds webhook as a policy source: external systems push documents to a receiver endpoint, which stages the files locally and triggers the policy run - Requests are authenticated with HMAC signatures; receiver hardened with bounded body reads and server-minted IDs - Uses the same team-scoped IntegrationConfig connection model as the S3 source, with matching portal UI (source type, icon, wizard) - Includes a policies-gated Cucumber feature covering the receiver end-to-end --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
dependabot[bot]andLudy
|
b5b9cc443f |
build(deps): bump com.github.junrar:junrar from 7.5.10 to 7.6.0 in /app/common (#7090)
Bumps [com.github.junrar:junrar](https://github.com/junrar/junrar) from 7.5.10 to 7.6.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/junrar/junrar/releases">com.github.junrar:junrar's releases</a>.</em></p> <blockquote> <h2>Release v7.6.0</h2> <h2>Changelog</h2> <h2>🚀 Features</h2> <ul> <li>support random access for files in solid RAR4 archives (<a href="https://github.com/junrar/junrar/commits/e0874d2">e0874d2</a>)</li> </ul> <h2>🏎 Perf</h2> <ul> <li>replace RarCRC.checkCrc with java.util.zip.CRC32 (<a href="https://github.com/junrar/junrar/commits/5270d23">5270d23</a>)</li> </ul> <h2>🛠 Build</h2> <p><strong>deps</strong></p> <ul> <li>bump gradle-wrapper to 9.5.1 (<a href="https://github.com/junrar/junrar/commits/cb4b7fd">cb4b7fd</a>)</li> <li>bump com.fasterxml.jackson.core:jackson-databind (<a href="https://github.com/junrar/junrar/commits/0bb56b3">0bb56b3</a>)</li> <li>bump com.fasterxml.jackson.datatype:jackson-datatype-jsr310 (<a href="https://github.com/junrar/junrar/commits/ca621b2">ca621b2</a>)</li> <li>bump org.jreleaser from 1.23.0 to 1.24.0 (<a href="https://github.com/junrar/junrar/commits/90f0548">90f0548</a>)</li> <li>bump commons-io:commons-io from 2.21.0 to 2.22.0 (<a href="https://github.com/junrar/junrar/commits/83a5d08">83a5d08</a>)</li> <li>bump com.github.ben-manes.versions from 0.53.0 to 0.54.0 (<a href="https://github.com/junrar/junrar/commits/d5abcdb">d5abcdb</a>)</li> </ul> <p><strong>unscoped</strong></p> <ul> <li>replace deprecated action (<a href="https://github.com/junrar/junrar/commits/338efcb">338efcb</a>)</li> </ul> <h2>Contributors</h2> <p>We'd like to thank the following people for their contributions: Gauthier, Gauthier Roebroeck, Robin Schimpf</p> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/junrar/junrar/blob/master/CHANGELOG.md">com.github.junrar:junrar's changelog</a>.</em></p> <blockquote> <h1><a href="https://github.com/junrar/junrar/compare/v7.5.10...v7.6.0">7.6.0</a> (2026-05-13)</h1> <h2>🚀 Features</h2> <ul> <li>support random access for files in solid RAR4 archives (<a href="https://github.com/junrar/junrar/commits/e0874d2">e0874d2</a>)</li> </ul> <h2>🏎 Perf</h2> <ul> <li>replace RarCRC.checkCrc with java.util.zip.CRC32 (<a href="https://github.com/junrar/junrar/commits/5270d23">5270d23</a>)</li> </ul> <h2>🛠 Build</h2> <p><strong>deps</strong></p> <ul> <li>bump gradle-wrapper to 9.5.1 (<a href="https://github.com/junrar/junrar/commits/cb4b7fd">cb4b7fd</a>)</li> <li>bump com.fasterxml.jackson.core:jackson-databind (<a href="https://github.com/junrar/junrar/commits/0bb56b3">0bb56b3</a>)</li> <li>bump com.fasterxml.jackson.datatype:jackson-datatype-jsr310 (<a href="https://github.com/junrar/junrar/commits/ca621b2">ca621b2</a>)</li> <li>bump org.jreleaser from 1.23.0 to 1.24.0 (<a href="https://github.com/junrar/junrar/commits/90f0548">90f0548</a>)</li> <li>bump commons-io:commons-io from 2.21.0 to 2.22.0 (<a href="https://github.com/junrar/junrar/commits/83a5d08">83a5d08</a>)</li> <li>bump com.github.ben-manes.versions from 0.53.0 to 0.54.0 (<a href="https://github.com/junrar/junrar/commits/d5abcdb">d5abcdb</a>)</li> </ul> <p><strong>unscoped</strong></p> <ul> <li>replace deprecated action (<a href="https://github.com/junrar/junrar/commits/338efcb">338efcb</a>)</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/junrar/junrar/commit/cb4b7fdb84269e95741fa22120e08ceb15a06152"><code>cb4b7fd</code></a> build(deps): bump gradle-wrapper to 9.5.1</li> <li><a href="https://github.com/junrar/junrar/commit/0bb56b3df4e3b6d14f73de92746a2eb47dd2d7a3"><code>0bb56b3</code></a> build(deps): bump com.fasterxml.jackson.core:jackson-databind</li> <li><a href="https://github.com/junrar/junrar/commit/ca621b22421f9e846df23112b32113230610c8d9"><code>ca621b2</code></a> build(deps): bump com.fasterxml.jackson.datatype:jackson-datatype-jsr310</li> <li><a href="https://github.com/junrar/junrar/commit/e0874d213832bbeaa8eb265c79479b01b5fa7392"><code>e0874d2</code></a> feat: support random access for files in solid RAR4 archives</li> <li><a href="https://github.com/junrar/junrar/commit/90f0548c728b0bf2e94f6e26f9e6396d9d0a7262"><code>90f0548</code></a> build(deps): bump org.jreleaser from 1.23.0 to 1.24.0</li> <li><a href="https://github.com/junrar/junrar/commit/83a5d085dc84991226229fef1515cc902d129241"><code>83a5d08</code></a> build(deps): bump commons-io:commons-io from 2.21.0 to 2.22.0</li> <li><a href="https://github.com/junrar/junrar/commit/338efcb5472be051989fbafd7179d561c71722eb"><code>338efcb</code></a> ci: replace deprecated action</li> <li><a href="https://github.com/junrar/junrar/commit/5270d235ade54d96dfc9958ab06f495fbbd169e7"><code>5270d23</code></a> perf: replace RarCRC.checkCrc with java.util.zip.CRC32</li> <li><a href="https://github.com/junrar/junrar/commit/d5abcdb9af988ddfa76e98d80787e15525488332"><code>d5abcdb</code></a> build(deps): bump com.github.ben-manes.versions from 0.53.0 to 0.54.0</li> <li><a href="https://github.com/junrar/junrar/commit/edadb28896962fdcf754029bafd0994aaff24530"><code>edadb28</code></a> chore(release): 7.5.10 [skip ci]</li> <li>See full diff in <a href="https://github.com/junrar/junrar/compare/v7.5.10...v7.6.0">compare view</a></li> </ul> </details> <br /> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Ludy <Ludy87@users.noreply.github.com> |
||
|
briosandAnthony Stirling
|
e2ea720fc8 |
refactor(api): replace regex literals with compiled patterns for improved performance and readability (#6511)
# Description of Changes This pull request refactors several utility classes and controllers to replace inline regular expression usage with precompiled `Pattern` constants. This change improves performance, consistency, and maintainability by ensuring that regex patterns are compiled only once and reused throughout the codebase. Additionally, it enhances code clarity and security in filename and SQL content sanitization. <!-- Please provide a summary of the changes, including: - What was changed - Why the change was made - Any challenges encountered Closes #(issue_number) --> --- ## Checklist ### General - [X] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [X] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [X] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [X] I have performed a self-review of my own code - [X] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [X] I have run `task check` to verify linters, typechecks, and tests pass - [X] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. --------- Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com> |
||
|
Anthony Stirling
|
79dc7d5615 |
Multi node cluster fixes (#7025)
- Exclude DataRedisRepositoriesAutoConfiguration (cluster crash-loop fix) - Share JWT signing keys via the DB + require a shared credential key in cluster mode - Make policy run status/listing visible across nodes --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
LudyandCopilot
|
bb830e5711 |
build: upgrade google-java-format and restore strict Spotless validation (#7091)
# Description of Changes - Upgraded google-java-format from 1.28.0 to 1.35.0. - Removed the broad `suppressLintsFor` workaround for the `google-java-format` step. - Ensured the shared `gradle/spotless.gradle` configuration is recognized by the relevant CI path filters and repository automation. - Kept the shared formatter configuration available to all backend modules. - Verified that google-java-format 1.35.0 runs successfully on JDK 25 for the Common, Core, and SaaS modules. - Confirmed that the previous claim about a general Guava 32.x crash on JDK 24/25 no longer justifies suppressing all formatter lint failures. ### Verification Verified with Temurin JDK 25.0.3 and google-java-format 1.35.0. The formatter still depends on Guava 32.1.3-jre, and no `suppressLintsFor` configuration is present. ```bash ./gradlew \ :common:spotlessJavaCheck \ :stirling-pdf:spotlessJavaCheck \ --rerun-tasks ``` Result: ```text > Task :common:spotlessJava > Task :common:spotlessJavaCheck > Task :stirling-pdf:spotlessJava > Task :stirling-pdf:spotlessJavaCheck BUILD SUCCESSFUL in 26s 4 actionable tasks: 4 executed ``` Using `--rerun-tasks` ensured that the formatter was executed and that the result did not come from the Gradle task cache. --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |
||
|
James Brunton
|
ed58d90ab8 |
Remove policies feature flag (#7031)
# Description of Changes Removes the feature flags for enabling policies on both the backend and frontend. We shouldn't be releasing another self-hosted release that doesn't include policies, so it makes sense to do this now. Builds that don't have the Processor will just not run policies because they won't have any. Beyond that, the API should always be available, but checks whether the user actually has the entitlements to run policies (whether they have credits/a payment method available) |
||
|
Ludy
|
d4edff9059 | fix(temp-files): prevent cleanup of active registered directories (#7006) | ||
|
Anthony Stirling
|
5944cd106b | Portal audit: label policy runs by their policy, flag automation sub-steps (#6937) | ||
|
ConnorYoh
|
ce6abe6e23 |
PAYG: size-scaled units + per-input-file PDF count + run-id grouping (#6957)
Reworks the Processor (PAYG) meter to **size-scaled units** while
keeping a true **PDF count** visible and distinct from units, and
replaces the fragile content+time lineage grouping with **explicit
per-run grouping**. Built as one PR across three slices.
> Status: **all three slices committed + verified.** `:saas` payg suite
green (418 tests, 0 failures); FE green (typecheck 0, 1260 tests, lint
0, format 0). Remaining before it takes effect in prod: run the
size-scaled default-policy SQL (below) in the Supabase SQL editor +
attach the $0.01/unit Stripe price.
## Model (what we're implementing)
- **Size scaling**: 1 unit per 50 MiB (bytes only, no page charge, no
cap). *(policy-row config, applied separately via SQL.)*
- **Charge = number of input files**: split (1→N outputs) = 1 charge;
merge (N→1) = N charges. `doc_count` = input files, fixed at open;
joined steps add 0.
- **Grouping by run id, not time**: a pipeline/policy/AI run = one
`run_id`; its tool sub-steps group into one charge (content-lineage
still maps split/merge journeys *within* the run). Two separate runs on
identical bytes = two charges. The 5-min window survives only as a
stale-job janitor.
- **10-tool split kept**: within a run's single-file lineage, an 11th
tool run opens a 2nd charge (step limit 10).
- **Count vs units surfaced**: usage page shows unique PDFs,
per-category (automation/AI/API) counts + units, and how many PDFs hit a
size multiplier with avg units/PDF.
## Slice 1 — run-id grouping (behavioural core)
- `AutomationRunContext` (common) — thread-scoped run id.
- `InternalApiClient` — stamps `X-Stirling-Run-Id`.
- Orchestrators open a run scope **on the worker thread that
dispatches** (async-safe): `PipelineProcessor.runPipelineAgainstFiles`,
`PolicyEngine.runToCompletion` (uses `run.getRunId()`),
`AiWorkflowService.orchestrate`.
- `ChargeContext` + `JobContext`: add `runId`; the charge interceptor
reads the header.
- `JobService.joinOrOpen`: `runId == null` → always open fresh
(standalone never joins); non-null → match scoped to the same `run_id`.
`JpaJobLineageStore`/`JobArtifactHashRepository`: add `run_id` filter to
the match query. Step-limit 10 unchanged.
## Slice 2 — doc_count + document_fingerprint
- V33 migration + entity fields.
- `JobService.openFresh`: set `docCount = inputs.size()`, compute
`document_fingerprint` from input signatures, and denormalise both onto
the DEBIT row in `JobChargeService.recordLedgerDebit`.
## Slice 3 — usage analytics API + FE
- `WalletLedgerRepository`: per-category `SUM(units)` +
`SUM(doc_count)`, `COUNT(DISTINCT document_fingerprint)`, and count of
rows whose units exceed their doc_count (a size multiplier fired), over
the period.
- `WalletSnapshotResponse` + `PaygWalletController`: add `categoryDocs`,
`docsProcessedThisPeriod`, `uniquePdfsThisPeriod`,
`sizeMultiplierPdfsThisPeriod`.
- FE `types.ts` + `PdfsProcessedCard` + `useWallet` + `walletFixtures` +
i18n: headline is the **PDF count**; a summary line shows "{unique}
unique · {units} meter units · {avg} avg units/PDF"; the split bar is
per-category PDF counts; a size-multiplier line shows how many PDFs
scaled. Count is separated from meter units so a 5-unit large PDF reads
as "1 PDF, 5 units".
## Config (out of PR — run in the Supabase SQL editor)
Wrap in one transaction. The partial-unique `is_default` index only
allows one default, so the old default is flipped off **before** the new
one is inserted. The new policy carries the prior default's
`free_tier_units` forward (change the literal if the launch grant should
differ).
```sql
BEGIN;
-- 1) flip default off the current policy + close its effective window
UPDATE stirling_pdf.pricing_policy
SET is_default = FALSE, effective_to = now()
WHERE is_default = TRUE;
-- 2) new default: 1 unit / 5 MiB, no page charge, no scaling cap.
-- free_tier_units carried from whatever the last policy granted (COALESCE→0).
INSERT INTO stirling_pdf.pricing_policy
(version, effective_from, doc_pages_per_unit, doc_bytes_per_unit,
min_charge_units, file_unit_cap, free_tier_units, is_default, notes, created_by)
VALUES
('v2-size-scaled-2026-07', now(),
2147483647, -- doc_pages_per_unit = INT_MAX → pages never drive units
52428800, -- doc_bytes_per_unit = 50 MiB → +1 unit per 50 MiB
1, -- min_charge_units
2147483647, -- file_unit_cap = INT_MAX → no cap on size scaling
COALESCE((SELECT free_tier_units FROM stirling_pdf.pricing_policy
ORDER BY effective_from DESC LIMIT 1), 0),
TRUE, 'Size-scaled: 1 unit/5MiB, bytes only, no cap', 'connor');
-- 3) per-source step limits: standalone ops = own charge; pipelines split at 10
INSERT INTO stirling_pdf.pricing_policy_step_limit (policy_id, job_source, step_limit)
SELECT p.policy_id, s.src, s.lim
FROM stirling_pdf.pricing_policy p
CROSS JOIN (VALUES
('WEB',1),('API',1),('DESKTOP_APP',1),('LINKED_INSTANCE',1),('PIPELINE',10)
) AS s(src, lim)
WHERE p.version = 'v2-size-scaled-2026-07';
-- 4) attach the $0.01/unit Stripe price (you handle the real price id)
INSERT INTO stirling_pdf.pricing_policy_stripe_price (policy_id, stripe_price_id)
SELECT policy_id, 'price_XXXXXXXX'
FROM stirling_pdf.pricing_policy WHERE version = 'v2-size-scaled-2026-07';
COMMIT;
```
Note: the `free_tier_units` subquery reads the most-recent policy
*before* the insert — run it as written (the new row doesn't exist yet
at step 2's SELECT).
## Self-hosted parity — tracked follow-up (not in this PR)
Combined-billing (`stirling.billing.account-link.enabled`) is a
**separate metering engine** (`app/proprietary/accountlink` —
`UsageMeterService`/`LocalUsageService`/`UsageSyncService`). The unit
*math* is shared (`DocumentUnitCalculator`), so size scaling matches
once the policy is pushed. But run-id grouping, `doc_count`, and
fingerprints must be mirrored there, and the usage-sync protocol
extended to report counts/fingerprints, before the self-hosted usage
page shows the same breakdown. Frozen/deferred, so this PR does SaaS;
self-hosted mirrors when it ships.
|
||
|
James Brunton
|
5ccb56da2d |
Add S3 policy source (#6948)
# Description of Changes * Adds an Amazon S3 Source & Output * Removes folder source from SaaS * Some miscellaneous UX fixes around pipelines |
||
|
ConnorYoh
|
119eb1f5ad |
Portal: move the admin route from /portal to /processor (#6933)
## What this changes
Moves the admin portal's browser route from **`/portal`** to
**`/processor`**, to match the "Processor" product name (the in-app app
switcher already says "processor").
- `PORTAL_BASENAME` `"/portal"` → `"/processor"` — the single source of
truth for all portal paths on the frontend, so every `toPortalPath(...)`
link and redirect follows automatically.
- `adminRouteExtensions` now mounts at `` `${PORTAL_BASENAME}/*` ``
instead of a hardcoded `"/portal/*"`, so the route can't drift from the
constant.
- **Backend:** `RequestUriUtils.isStaticResource` now treats
`/processor` (not `/portal`) as the SPA shell, so a direct navigation or
hard refresh to `/processor` serves the app instead of 404ing. (This is
the only backend URL reference — there's no Spring Security matcher for
it.)
- Updated the two portal route tests and the doc comments that named the
old path.
**Not changed:** the `@portal/*` import alias — that's the code layer /
flavor-layering path, not the URL. Renaming it would be a much larger,
unrelated churn.
The portal isn't publicly launched yet, so there are no existing links
to preserve — no redirect from the old path is included.
## Testing
- `task frontend:typecheck:all`, full test suite (1,211), lint, format —
green
- `./gradlew :common:test --tests "*RequestUriUtilsTest"` — green
|
||
|
EthanHealy01
|
c64369e56c |
Classifier Policy (#6898)
## Overview Adds **AI document classification** and a **classification-aware Files sidebar**: uploaded documents are automatically tagged with document-type labels (Invoice, Contract, Lab report, …), and the sidebar groups files under editable parent categories so a large library stays navigable. > [!IMPORTANT] > **This feature only runs in the SaaS build.** Classification depends on the AI engine and team-scoped label storage, so it's gated to SaaS end-to-end: > - The sidebar grouping is a `saas/`-layer override of the `fileSidebarGrouping` seam; every other build (OSS core, self-hosted proprietary, desktop) gets the null stub and renders the **unchanged flat, recency-sorted list** — no categories, no "Other", no picker. > - The classify/labels backend endpoints are gated on `policies.enabled` (on in SaaS) and live in `app/proprietary`, so they're absent from pure OSS and dormant in self-hosted unless explicitly enabled. > - The Python classifier is reached only via that gated path. > > Shared-layer changes that do compile everywhere are inert without the engine (dormant schema/field additions) or intentional (`GetInfoOnPDF` surfacing custom metadata). ## What it does - **Classifier (engine):** reads the first/last two pages of a PDF and assigns document-type labels from an allowed vocabulary. Labels are deliberately document-*type* descriptors — no deep-content/PII detection, since only a page window is read. - **Team label vocabulary:** ~270 built-in defaults across ~15 families, seeded per team. Editable by team leaders/admins in the Classification policy settings (import/export/reset). Team-scoped and shared; **per-user personal labels are intentionally out of scope** — the vocabulary is team-level only. - **Sidebar categories:** files group under parent categories (Financial, Legal, Medical, …), busiest-first, collapsible, with a "Recent" group on top and an "Other" group for anything uncategorised. The category structure (names, icons, membership, custom categories) is **device-local and user-editable** via a "Customize" picker — the only per-user personalization; it never changes the team's label vocabulary. - Classification results are written to PDF metadata (`StirlingPDFClassification`), read back to keep files in their groups without re-parsing. ## Architecture Spans all three layers, mirroring the existing policy/source subsystem conventions: - **`frontend/editor`** — sidebar grouping seam + SaaS override, category manager, labels editor, icon palette, file grouping, tests, `en-US` i18n. - **`app/proprietary` + `app/common` + `app/core`** — `ClassifyLabelController`, team-scoped `ClassificationLabelStore` (Jpa + in-process impls, same shape as `PolicyStore`/`SourceStore`), metadata read/write. - **`engine`** — the document-classifier agent, contracts, routes, tests. ## Screenshots **Files sidebar — grouped by category (SaaS)** ### Loading view <img width="2056" height="1046" alt="Screenshot 2026-07-07 at 5 12 56 PM" src="https://github.com/user-attachments/assets/1d712da5-50ae-4349-b0cd-e62665c3ec0c" /> ### Organized in the sidebar <img width="2056" height="1045" alt="Screenshot 2026-07-07 at 5 14 05 PM" src="https://github.com/user-attachments/assets/3ea4fe21-da51-4cea-bc3a-18ce040d3d05" /> **Customize categories picker** ### Personal settings to change how labels are grouped in an individual users editor <img width="2056" height="1044" alt="Screenshot 2026-07-07 at 5 52 42 PM" src="https://github.com/user-attachments/assets/40be03ce-0f63-4d1e-b58b-cec045d01cb2" /> **Classification labels editor (team settings)** <img width="2056" height="1042" alt="Screenshot 2026-07-07 at 5 53 00 PM" src="https://github.com/user-attachments/assets/337b0739-15c9-4749-9c6b-22e3b20825b8" /> ## Testing - Frontend `task frontend:check` — green (editor + portal tests, typecheck across all flavors, lint, label-drift guard). - Backend `task backend:check` (proprietary) and `:saas:test` — green. - Engine `task engine:check` — green. |
||
|
Anthony Stirling
|
0692058602 |
Embed admin portal as its own app in the jar behind buildWithPortal (#6911)
## What
Lets the admin portal ("Stirling Processor") ship **inside the JAR**,
gated by a build flag. On `main` the portal already exists as a lazy
`/portal/*` route in the editor but isn't included in production builds
and isn't reachable in a login-enabled server. This PR makes it a
**flag-gated, directly-navigable** part of the editor bundle, and wires
it into the PR preview deployment so it can be tried live.
It keeps the exact architecture `main` uses (portal = a lazy chunk of
the editor, not a separate app), so it inherits all the editor's global
providers/styles and there's no second build to maintain.
## How
**Frontend - gate the existing lazy route**
([`adminRouteExtensions.tsx`](frontend/editor/src/proprietary/routes/adminRouteExtensions.tsx))
```ts
const includePortal = import.meta.env.VITE_INCLUDE_PORTAL === "true" || import.meta.env.DEV;
const PortalApp = includePortal ? lazy(() => import("@portal/PortalApp")) : null;
```
Vite bakes the env to a literal, so when off the dynamic import is
**tree-shaken out entirely** (no `PortalApp` chunk emitted). Always on
in dev. `VITE_INCLUDE_PORTAL` is typed in `vite-env.d.ts` and declared
(default `false`) in `editor/.env`.
**Gradle** ([`build.gradle`](app/core/build.gradle)) -
`-PbuildWithPortal=true` forces `buildWithFrontend=true` and sets
`VITE_INCLUDE_PORTAL=true` on the editor build. Process-env takes
priority over `.env`, so the flag wins for JAR builds while plain `vite
build` / Cloudflare Pages default to off.
**Backend - make the shell reachable**
([`RequestUriUtils`](app/common/src/main/java/stirling/software/common/util/RequestUriUtils.java))
- permits `/portal` + `/portal/*` as public SPA routes. The editor keeps
its JWT in localStorage (not a cookie), so a direct nav/refresh to
`/portal` isn't authenticated at the server and would otherwise redirect
to `/login` and never load. Serving the shell pre-auth (like the editor
root already is) lets it load; **access control is unchanged** - the
portal has its own auth gate + `RequirePortalAccess`, and its data APIs
stay protected.
**Docker** - the embedded Dockerfiles take `ARG BUILD_PORTAL=false` →
`-PbuildWithPortal=${BUILD_PORTAL}`. Default off, so official
`push-docker` images do **not** bundle the portal.
**CI - scoped to the PR preview deploy only**
([`PR-Auto-Deploy-V2.yml`](.github/workflows/PR-Auto-Deploy-V2.yml)) -
the one job that builds the JAR and comments owns all portal wiring:
passes `BUILD_PORTAL=true`, enables the portal's backend features
(`POLICIES_ENABLED`, `STIRLING_BILLING_ACCOUNT_LINK_ENABLED`), and adds
an "Admin portal included" line (linking `/portal` via the direct IP) to
the deployment comment. `push-docker`, `build.yml`, `test-build-docker`,
and the shared paths-filter are untouched.
## Validation (real, in the JAR)
Built and booted the JAR with `-PbuildWithPortal=true` and login
enabled:
- `/portal` and `/portal/users` load via direct nav and render **fully
themed** (dark surfaces, gradients, filled buttons).
- Editor-only build (`-PbuildWithFrontend=true`, no portal flag) →
editor ships, **0 portal chunks** (tree-shaken).
- `-PbuildWithPortal=true` → `PortalApp` chunk present.
Green: `frontend:check:all` (typecheck all variants, lint, format,
build, tests incl. the `VITE_*` env guard), backend compile,
`RequestUriUtilsTest`, spotless.
## Notes
- **Official images never bundle the portal** (Dockerfile default off);
only the PR preview does. Flip `BUILD_PORTAL` / `-PbuildWithPortal` to
include it elsewhere.
- The `/portal` shell being public is the one deviation from `main`, and
it's required for the route to be reachable at all in a login-enabled
server; data access is still fully gated.
|
||
|
Peter Dave HelloandJames Brunton
|
16cfbc170e |
Clean up typos in docs, comments, and UI copy (#6045)
# Description of Changes Fix wording, numbering, path references, and minor grammar issues across project guides, backend comments, and frontend strings. This keeps documentation and user-facing text consistent without changing application behavior. --- ## Checklist ### General - [x] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md#6-testing) for more details. Co-authored-by: James Brunton <jbrunton96@gmail.com> |
||
|
Anthony Stirling
|
84739e8b0e |
Align settings.yml defaults and fix dead/mismapped settings (#6816)
# Description of Changes Align settings.yml defaults and fix dead/mismapped settings --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
James Brunton
|
3f7e898c69 |
Add sources service and frontend (#6774)
# Description of Changes Redesign policies backend to treat sources a lot closer to how the frontend imagined them working (they're persistent now and have an API). Then connect the portal to the sources when mocks are off to allow for source creation in the UI. It's not particularly useful to do that right now because there's no policies UI, but I've tested manually that sources set up in the UI are usable by policies created via the API. I had to change the portal so that when mocks are off, it doesn't just hard crash when attempting to connect to all the backend APIs that don't exist yet. It'll still log the errors, but just continues on rendering the UI now. I also changed all the policies backend APIs to be gated behind a flag instead of behind the SaaS profile. This is because we haven't yet got the payment model sorted, but we're going to need this stuff running self-hosted to be able to test it locally. |
||
|
Anthony Stirling
|
bc6f1a1ff5 | Add login agreement disclaimer feature (#6766) | ||
|
Anthony Stirling
|
d06d3cabaf | chore: update svg conversion and database import handling (#6796) | ||
|
Anthony Stirling
|
f7f7b8790e |
fix update notification visibility and install flicker (#6776)
# Description of Changes - Closes #6754 - Update popup now hidden on mobile, for non-admins, and never on SaaS - Respects admin "Show Update Notifications" setting (`showUpdate` / `showUpdateOnlyAdmin`, now default on) - Fixes update modal flickering during desktop install --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
Ludy
|
e35594f946 |
chore(build): centralize Gradle dependency version management (#6499)
# Description of Changes This change centralizes several dependency version declarations into shared Gradle version properties and updates module build files to reference those properties instead of hardcoded version strings. ### What was changed - Added centralized version properties in the root `build.gradle` for: - commons-io - commons-lang3 - rhino - okhttp BOM - gson - guava - bucket4j - archunit - batik - jpdfium - JWT - AWS SDK - Testcontainers - Replaced hardcoded dependency versions across multiple modules with shared version variables. - Updated `resolutionStrategy.force` declarations to use centralized version properties. - Updated dependency constraints and BOM references to use shared version variables. - Removed module-specific duplicate version declarations from `app/proprietary/build.gradle`. - Standardized dependency declarations across `common`, `core`, `proprietary`, and `saas` modules. ## Why the change was made - Reduce duplication of dependency version definitions. - Simplify future dependency upgrades and maintenance. - Ensure consistent dependency versions across all modules. - Improve readability and reduce the risk of version drift between subprojects. - Make security-related dependency overrides easier to maintain from a single location. --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
Anthony Stirling
|
0a29186ed6 |
Add JUnit tests to raise coverage across all modules (#6782)
# Description of Changes AI generated junit tests --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
Anthony Stirling
|
3870ac3d7d |
Add desktop mobile-upload page and fix LAN QR URL (#6736)
# Description of Changes Desktop can not use QR code upload due to API backend not having UI for it... Because of this we add UI, has to be custom because can not support OpenCV and in app camera due to its https requirement <img width="919" height="2048" alt="image" src="https://github.com/user-attachments/assets/d84c3903-fd23-421a-8919-24d89ba6c753" /> <img width="919" height="2048" alt="image" src="https://github.com/user-attachments/assets/d6c8fdf3-837b-4dc3-92ea-37f7502f8462" /> <img width="919" height="2048" alt="image" src="https://github.com/user-attachments/assets/fd4ce3b4-69ad-45dd-b066-bff2d082c583" /> <img width="1550" height="790" alt="image" src="https://github.com/user-attachments/assets/701d4dcc-ebd6-4e03-aa7b-2d8623525fc9" /> --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
Anthony Stirling
|
f33f4f8f75 |
Add JUnit tests for common and core module coverage (#6675)
# Description of Changes JUNITS! They JUnits were 100% AI generated however no code was touched --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
|
fb6a118be9 |
Update SaaS to latest main (#6667)
# Description of Changes > [!warning] > **Do not** squash this on merge. It should be merged via a merge commit Fixes conflicts in `pgvector_store.py`. Also since codespell is failing, add comments to ignore the errors in `sync_en_us_spelling.py` --------- Co-authored-by: Ludy <Ludy87@users.noreply.github.com> Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com> |
||
|
James Brunton
|
04d68c650a |
Merge remote-tracking branch 'origin/main' into saas-update
# Conflicts: # engine/src/stirling/documents/pgvector_store.py |
||
|
Anthony Stirling
|
9d7467cf90 | Scope signing user picker to team for multi-tenant SaaS (#6583) | ||
|
Ludy
|
1ce765ab1e | refactor: replace legacy Paths usage with Path.of (#6441) | ||
|
James Brunton
|
d995471a55 |
Merge remote-tracking branch 'origin/main' into SaaS-update
# Conflicts: # frontend/editor/src/proprietary/components/chat/ChatContext.tsx # frontend/editor/src/saas/components/shared/TrialStatusBanner.tsx |
||
|
Anthony Stirling
|
1d598d5caa | MCP OAuth discovery fix + Supabase consent page (#6608) | ||
|
ConnorYohandReece
|
cf513c255b |
PAYG: pay-as-you-go billing — metered automation/AI/API + one-time free grant (#6589)
## Summary Pay-as-you-go (PAYG) billing for Stirling-PDF SaaS. Manual PDF editing stays free forever; only **automation, AI, and API** usage is metered. Every team gets a **one-time lifetime free grant** (default 500 PDFs) before any billing; past that, a team adds a card and pays per metered document, with a self-set monthly spending cap. This branch combines and supersedes the in-flight BE (#6574) and FE (#6579) work plus the SaaS edge functions (Stirling-PDF-SaaS PR, now on `v3`), hardened into a single reviewable feature after a pre-merge dead-code/security review. ## Billing model - **Always free:** manual / JWT web-tool usage is `BYPASSED` — never metered, no matter where it's triggered. - **Billable categories:** `AUTOMATION`, `AI`, `API`. - **One-time lifetime free grant** (`pricing_policy.free_tier_units`, default 500): never resets, survives subscribing. It gates unsubscribed teams (billable API calls hard-stop with a 402 once exhausted) and decides the free-vs-paid split of every job. - **Subscribed:** paid documents (beyond the grant) are metered to a Stripe Billing Meter; an optional monthly spending cap degrades billable categories when reached. - **Dedup:** the same file pushed through several steps within a workflow window counts **once** (lineage join), so API/AI chaining on one file isn't double-charged. ## What's included **Database** — Flyway migrations `V11`→`V21` with matching Supabase twins: pricing policy + per-team sidecar (`payg_team_extensions`: subscription id, Stripe customer, free-grant counter), append-only `wallet_ledger`, shadow charges, subscription-state RPCs (`V14`), audit logs (`V15`), billing category (`V16`), one-time lifetime free grant (`V19`), launch-grant seed (`V20`), drop of the unused `wallet_category_summary` view (`V21`). **Charge pipeline** — `PaygChargeInterceptor` (open/join a process, split the free grant, write the ledger DEBIT), `JobChargeService` (consume the grant under a row lock, restore it on a first-step refund, meter only the paid portion on completion), `StaleJobCloser` fallback (idempotent close → meter). **Entitlement** — `EntitlementService` (per-team cached snapshot: grant-gated for free teams, monthly-cap-gated for subscribed) + `EntitlementGuard` (401 `SIGNUP_REQUIRED` / 402 `FEATURE_DEGRADED` / `PAYG_LIMIT_REACHED`). **Metering** — `PaygMeterReportingService` writes a durable `payg_meter_event_log` row around every POST to the `meter-payg-units` edge fn (pending → posted/failed); `PaygMeterReconcileScheduler` retries unposted events under the same idempotency key inside Stripe's 24h dedup window. **Billing facts** — `TeamBillingService` reads the synced `stripe.*` mirror (subscription window, per-document rate; the unsubscribed-team estimate resolves the rate by Price `lookup_key = plan:processor`). **Wallet API** — `PaygWalletController`: `GET /api/v1/payg/wallet`, `PATCH /api/v1/payg/cap`. **Frontend** — PAYG Plan page (two-card free layout + subscribed views), `useWallet`, upgrade modal with lazy-loaded Stripe Embedded Checkout and a shared `SpendCapControl`, customer-portal link, 402/401 interceptor toast, en-GB i18n. (Per-member usage shows each teammate's spend; the activity feed is behind a flag until polished.) **SaaS edge functions** (`Stirling-PDF-SaaS` `v3`) — `create-checkout-session`, `create-payg-team-subscription`, `create-customer-portal-session`, `meter-payg-units`, `payg-subscription-webhook`, `stripe-sync`, plus the stripe-sync `migrate` + scoped-`backfill` scripts. All price lookup is DB-driven (no `STRIPE_PAYG_PRICE_ID_*` env vars). ## Release prerequisites (prod) 1. Apply Flyway migrations (`V11`→`V21`) and the Supabase migration twins. 2. Stripe Sync Engine: run `stripe-sync:migrate`, then a **scoped** backfill — `product`, `price`, `customer`, `subscription` only (not `all`, which rate-limits). 3. Register 2 PAYG webhook endpoints (each its own signing secret): `stripe-sync` (product/price/customer/subscription `.*`) and `payg-subscription-webhook` (`customer.subscription.created`/`.deleted` drive state; `.updated` + `invoice.*` observed). Keep the legacy `stripe-webhook` only if credits/self-hosted flows still run. 4. Stripe Billing Meter: `event_name = payg_doc_units`, value key `processed_documents`. 5. Env: `PAYG_METER_ENDPOINT` + `SUPABASE_EDGE_FUNCTION_SECRET` (backend); the webhook signing secrets (edge fns). The default pricing policy must point at the PAYG Stripe Price(s); `V20` seeds `free_tier_units = 500`. ## Testing - `:saas:test` green, `:saas:spotlessCheck` clean, edge-fn Deno tests green, FE saas typecheck clean (the remaining errors are pre-existing `proprietary/*` + `prototypes/*`, untouched here). Cucumber shadow-mode suite + CI workflow included. ## Pre-merge review An independent dead-code/security pass came back **clean on security** (team-derived authz / no IDOR, leader-only cap mutation, no billing-category downgrade, dev/mock hooks gated to `import.meta.env.DEV` + `/dev/`, no secrets/injection, fail-open metering by design). The dead/unwired code it flagged has been removed in this branch (unenforced sub-cap control, an unused JDBC DAO + its view, dead methods). ## Follow-ups (tracked, not blocking) - **Enforce per-member sub-caps** — the control was removed because it read for display but never gated a request; the per-member usage display and `cap_units` column are retained for when enforcement is wired. - **API/AI chaining billing model + `ProcessType` enum** — confirm same-file dedup covers API chaining; define per-tool AI charging; decide whether the unused enum values stay. - **Activity feed** — hidden behind a flag until the meter-event surface is polished. --------- Co-authored-by: Reece <reece@stirlingpdf.com> |
||
|
EthanHealy01andAnthony Stirling
|
88adb7adad |
create agent (#6520)
Added the create agent. Use [these prompts](https://github.com/Stirling-Tools/Stirling-PDF-SaaS/blob/main/docgen/backend/default_templates/sample_prompts.md) to test or try your own :) Here’s the one I use ``` Hey, I need to generate an employee expense report for reimbursement. Company: Summit Consulting Partners Company address: 88 Riverside Plaza, Suite 1400, New York, NY 10069 Accounting department email: expenses@example.com Employee details: * Employee Name: Michael Tran * Employee ID: EMP-1047 * Department: Client Services * Report Date: January 20th, 2026 * Reporting Period: January 5th, 2026 – January 16th, 2026 * Manager Approver: Laura Simmons Trip purpose: Client onsite meetings with Atlantic Energy Solutions in Boston, MA. Expense items: * Flight (NYC to Boston roundtrip) — $325.40 — January 5th, 2026 — Airline ticket * Hotel (3 nights at Harborview Hotel) — $822.75 — January 5th-8th, 2026 * Taxi from airport to hotel — $48.00 — January 5th, 2026 * Client dinner (3 attendees) — $186.20 — January 6th, 2026 * Parking at JFK Airport — $72.00 — January 5th-8th, 2026 * Breakfast (per diem not used) — $18.50 — January 7th, 2026 * Uber to client office — $22.10 — January 7th, 2026 * Printing + presentation materials — $46.90 — January 8th, 2026 * Lunch with client — $39.75 — January 8th, 2026 * Office supplies (notebooks, pens) — $27.60 — January 10th, 2026 * Mileage reimbursement (client visit in NJ, 42 miles @ $0.67/mile) — $28.14 — January 14th, 2026 * Team lunch meeting (internal) — $64.30 — January 15th, 2026 Reimbursement method should be direct deposit. Add a notes section stating: "All receipts attached. Expenses are business-related and comply with company travel policy." ``` --------- Co-authored-by: Anthony Stirling <77850077+frooodle@users.noreply.github.com> |
||
|
EthanHealy01
|
5fca2f199a | Feature/pdf ingestion jpdfium (#6525) | ||
|
James Brunton
|
3675db5907 |
Add new triggers, sources and sinks to policies (#6543)
# Description of Changes Add new triggers: - Schedule (fires every X amount of time) - Folder watch (fires whenever the OS tells us a folder has a new file in it; on Mac this is technically a 2s schedule but that's just how Java implements it) Add new sources: - Folder (reads from this directory) Add new sinks: - Inline (stores in FileStorage) - Folder (stores in specified directory) Still want to do S3 buckets and web hooks and stuff, but they can come in a future PR. I'm hoping this should make it sufficient to be able to integrate with processing folders frontend etc. I've also changed it so that policies can have multiple sources and triggers at once, which seems like it might be useful. |
||
|
Anthony Stirling
|
3ecd95b779 |
Add MCP server with OAuth/API-key auth (#6570)
Adds an optional MCP server (proprietary module) that exposes Stirling's PDF operations and AI capabilities to MCP clients. Off by default, zero footprint when disabled. ### What - New `/mcp` endpoint: streamable-HTTP + JSON-RPC 2.0; 8 tools (describe_operation, pages/convert/misc/security category tools, AI, upload, download). - Runs real operations over an internal loopback; results returned inline as base64 (small) or by fileId (large). ### Auth (two modes) - OAuth2 resource server: RFC 9728 protected-resource metadata, RFC 8707 audience binding, JWKS, `mcp.tools.read/write` scopes; binds each token to a provisioned Stirling account. - API-key mode: reuses Stirling per-user `X-API-KEY` (no IdP needed). ### Security - Per-user file ownership in FileStorage: async/queued writes scoped to the submitting user; legacy/owner-less files stay readable. - Admin allow/block list controls which operations are exposed. - Python engine gated behind a shared secret (`X-Engine-Auth`). - MCP filter chain is isolated and cannot weaken the main app's security. - Hardened: no upstream error-body leakage, log injection sanitized, fileId path/sidecar enumeration blocked. ### Config / footprint - Disabled by default (`mcp.enabled=false`); all beans `@ConditionalOnProperty`. --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
Anthony Stirling
|
71361f0d33 | Minor: Office doc changes (#6571) | ||
|
James Brunton
|
51478e5051 |
Policies backend (#6527)
# Description of Changes Add a backend for running any multi-step PDF operations. This is designed to be used for the upcoming Policies feature, along with anything else that will require automated running of PDF operations, like the Automate tool or Processing Folders. The implementation is not complete. I've tried to get all the infrastructure in there so that we can add in whichever triggers we need in the future (like cron triggers or watching folders on disk) but currently it just supports manual triggering of the policy. The basis of this work was the operation running from the Stirling Engine, which this PR removes in favour of this new system. The only currently accessible frontend way to test this work is to ask the AI chat to execute multiple operations on a PDF, but I've also extensively tested with direct API calls to make sure that the policies work and persist properly. |
||
|
EthanHealy01andJames Brunton
|
35a712a278 |
smart redaction (#6195)
Co-authored-by: James Brunton <jbrunton96@gmail.com> |
||
|
James Brunton
|
b705c5b84c |
Switch to use JPDFium v1.0.2, which signs the Mac binaries (#6521)
# Description of Changes Currently, it's not possible to develop the backend on Mac without manually signing the JPDFium binaries yourself since macOS will reject running the unsigned binaries. [We've now updated JPDFium to sign the Mac binaries in v1.0.2](https://github.com/Stirling-Tools/JPDFium/releases/tag/v1.0.2), so update to use that version. |
||
|
Anthony Stirling
|
b355ccec9e | Add Valkey cluster backplane and sticky-410 ownership (clusters) (#6472) |