Compare commits

...
Author SHA1 Message Date
EthanHealy01 282f18705d draft: Make Tool Steps Seperate Pages 2026-06-17 16:44:02 +01:00
Reece Browne f127d4f575 fix(policies): poll runs to completion with progress, soft-retry when queue is full (#6690)
## What & why

Production reports of policy enforcement "hanging" traced to
large/many-page documents: the watermark step's flatten-to-image
(`convertPDFToImage`) on a 500+ page PDF takes minutes, exceeding both
the client poll cap and the backend per-step timeout. This makes the
slow case graceful instead of looking broken, and makes load-shedding
non-fatal.

### Poll runs to completion (no false "hang")
The client poll loop used a flat ~150s cap that was **shorter than the
backend's 300s per-step timeout**, so it abandoned long-but-healthy runs
mid-flight. The budget is now sized to the backend's real worst case —
`stepCount × per-step timeout + grace`, learned from the first status
report — so the client always polls long enough to surface the run's
**actual** terminal state (success or the backend's real error) rather
than a misleading client-side timeout.

### Per-step progress
The activity feed now shows `Enforcing… · step n/m` (from
`currentStep`/`stepCount`), so a slow step shows movement instead of a
dead spinner.

### Soft-retry on queue rejection
Under load the shared `JobQueue` rejects runs ("queue full"), which
previously surfaced as a hard failure needing a manual Retry. The
backend now tags that rejection with a stable `POLICY_QUEUE_FULL`
errorCode; the client treats it as transient backpressure and
**auto-retries the file in place** with exponential backoff (≈4s→64s, ~2
min), shown as a soft "Busy — retrying…" row, falling back to the manual
Retry only once the retry budget is spent.

## Testing
- **Frontend unit tests** (30 pass across the policies suite), including
a new `usePolicyAutoRun.retry.test.tsx` that drives the real controller
orchestration (poll → `POLICY_QUEUE_FULL` → relabel → backoff → in-place
re-dispatch), plus poll-budget, step-progress, and activity-feed relabel
cases.
- **Backend** `PolicyEngineTest` case asserting a queue-rejected run
carries the `POLICY_QUEUE_FULL` code.
- Typecheck clean on all three flavors (proprietary/saas/core); prettier
+ spotless clean.
- Poll-budget + progress + real-error surfacing were also verified live
end-to-end against a 599-page run (survived past the old cap, showed
step progress, reported the backend's real 300s-timeout failure,
recovered after a simulated network drop).

## Not included (follow-ups)
- The underlying flatten-to-image cost itself (bounded-memory/streaming
flatten, revisiting `convertPDFToImage` default and the 300s timeout) —
the real perf fix, deliberately out of scope here.
2026-06-16 17:32:12 +00:00
Anthony Stirling f33f4f8f75 Add JUnit tests for common and core module coverage (#6675)
# Description of Changes

JUNITS!
They JUnits were 100% AI generated however no code was touched

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-06-16 16:23:34 +00:00
7e67bfc459 Fix SaaS issues (#6694)
# Description of Changes

Fixes several SaaS issues,  was integration branch for saas release

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: James Brunton <jbrunton96@gmail.com>
Co-authored-by: Reece Browne <74901996+reecebrowne@users.noreply.github.com>
Co-authored-by: ConnorYoh <40631091+ConnorYoh@users.noreply.github.com>
Co-authored-by: Reece <reece@stirlingpdf.com>
Co-authored-by: EthanHealy01 <80844253+EthanHealy01@users.noreply.github.com>
Co-authored-by: Ludy <Ludy87@users.noreply.github.com>
2026-06-16 17:16:07 +01:00
Anthony Stirling 686fb1fb50 Revert "SaaS fixes" (#6693)
Reverts Stirling-Tools/Stirling-PDF#6578 due to mistaken squash merge
not normal merge
2026-06-16 17:13:10 +01:00
Anthony Stirling 5389e39cfc Revert "SaaS fixes (#6578)"
This reverts commit ddf78d11ae.
2026-06-16 16:48:30 +01:00
ddf78d11ae SaaS fixes (#6578)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: James Brunton <jbrunton96@gmail.com>
Co-authored-by: Reece Browne <74901996+reecebrowne@users.noreply.github.com>
Co-authored-by: ConnorYoh <40631091+ConnorYoh@users.noreply.github.com>
Co-authored-by: Reece <reece@stirlingpdf.com>
Co-authored-by: EthanHealy01 <80844253+EthanHealy01@users.noreply.github.com>
Co-authored-by: Ludy <Ludy87@users.noreply.github.com>
2026-06-16 16:41:25 +01:00
James Brunton 10b4551449 Fix failing Playwright test in SaaS (#6688)
# Description of Changes
Fix Playwright failing test in SaaS (I think this is my third attempt
now so who knows if this will actually fix it for real this time, but
hopefully it does)
2026-06-16 15:56:30 +01:00
Reece Browne 96accea984 Portal: full mock-driven surfaces, demonolithed components, backend-ready mocks (#6686) 2026-06-16 12:20:35 +01:00
James Brunton 9a883be697 Cleanup of SaaS code (#6669)
# Description of Changes
De-AI comments and fix ridiculously indented code
2026-06-16 11:49:13 +01:00
dependabot[bot]andAnthony Stirling 6716398ccb build(deps): bump go-task/setup-task from 2.0.0 to 2.1.0 (#6429)
Bumps [go-task/setup-task](https://github.com/go-task/setup-task) from
2.0.0 to 2.1.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/go-task/setup-task/releases">go-task/setup-task's
releases</a>.</em></p>
<blockquote>
<h2>v2.1.0</h2>
<h2>What's Changed</h2>
<ul>
<li>Replaced <code>typed-rest-client</code> with
<code>@actions/http-client</code> for GitHub API calls
to eliminate the Node 24 <code>DEP0169</code> deprecation warning about
<code>url.parse()</code> (<a
href="https://redirect.github.com/go-task/setup-task/issues/5">#5</a> by
<a href="https://github.com/vmaerten"><code>@​vmaerten</code></a>).</li>
<li>Modernized the TypeScript tooling stack (vitest, oxlint,
<code>@actions/core@2</code>,
<code>@actions/io@2</code>, updated <code>@types/node</code>,
<code>@vercel/ncc</code>, <code>prettier</code>, etc.) (<a
href="https://redirect.github.com/go-task/setup-task/issues/5">#5</a> by
<a href="https://github.com/vmaerten"><code>@​vmaerten</code></a>).</li>
<li>Migrated the project to ESM (sources + bundle). Aligns with the new
<code>@actions/*</code> ESM-only majors and produces a ~47% smaller
<code>dist/index.js</code> (<a
href="https://redirect.github.com/go-task/setup-task/issues/5">#5</a> by
<a href="https://github.com/vmaerten"><code>@​vmaerten</code></a>).</li>
<li>Upgraded <code>@actions/core</code> 2 → 3,
<code>@actions/http-client</code> 2 → 4,
<code>@actions/io</code> 2 → 3, <code>@actions/tool-cache</code> 2 → 4,
<code>typescript</code> 5 → 6, and
<code>markdownlint-cli</code> 0.47 → 0.48 (<a
href="https://redirect.github.com/go-task/setup-task/issues/5">#5</a> by
<a href="https://github.com/vmaerten"><code>@​vmaerten</code></a>).</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/go-task/setup-task/blob/main/CHANGELOG.md">go-task/setup-task's
changelog</a>.</em></p>
<blockquote>
<h1>Changelog</h1>
<h2>Unreleased</h2>
<h2>v2.1.0 - 2026-05-17</h2>
<ul>
<li>Replaced <code>typed-rest-client</code> with
<code>@actions/http-client</code> for GitHub API calls
to eliminate the Node 24 <code>DEP0169</code> deprecation warning about
<code>url.parse()</code>.</li>
<li>Modernized the TypeScript tooling stack (vitest, oxlint,
<code>@actions/core@2</code>,
<code>@actions/io@2</code>, updated <code>@types/node</code>,
<code>@vercel/ncc</code>, <code>prettier</code>, etc.).</li>
<li>Migrated the project to ESM (sources + bundle). Aligns with the new
<code>@actions/*</code> ESM-only majors and produces a ~47% smaller
<code>dist/index.js</code>.</li>
<li>Upgraded <code>@actions/core</code> 2 → 3,
<code>@actions/http-client</code> 2 → 4,
<code>@actions/io</code> 2 → 3, <code>@actions/tool-cache</code> 2 → 4,
<code>typescript</code> 5 → 6, and
<code>markdownlint-cli</code> 0.47 → 0.48.</li>
</ul>
<h2>v2.0.0 - 2026-03-18</h2>
<ul>
<li><strong>BREAKING</strong>: Upgraded to Node 24. Requires a GitHub
Actions runner with
Node.js 24 support
(<a
href="https://redirect.github.com/go-task/setup-task/pull/10">#10</a> by
<a href="https://github.com/vmaerten"><code>@​vmaerten</code></a>).</li>
</ul>
<h2>v1.1.0 - 2026-03-17</h2>
<ul>
<li>Added configurable HTTP retry for API requests
(<a href="https://redirect.github.com/go-task/setup-task/pull/7">#7</a>
by <a
href="https://github.com/vmaerten"><code>@​vmaerten</code></a>).</li>
</ul>
<h2>v1.0.0 - 2025-09-12</h2>
<ul>
<li>Forked <a
href="https://github.com/arduino/setup-task">arduino/setup-task</a> (by
<a href="https://github.com/pd93"><code>@​pd93</code></a>).</li>
<li>Default <code>repo-token</code> to <code>{{github.token}}</code>
(<a
href="https://redirect.github.com/arduino/setup-task/pull/642">arduino/setup-task#642</a>
by
<a href="https://github.com/shrink"><code>@​shrink</code></a>).</li>
<li>Fixed a bug where the action would fail is Task pushed a tag without
a release
(<a
href="https://redirect.github.com/arduino/setup-task/pull/490">arduino/setup-task#490</a>,
<a
href="https://redirect.github.com/arduino/setup-task/pull/1193">arduino/setup-task#1193</a>
by
<a href="https://github.com/trim21"><code>@​trim21</code></a>).</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/go-task/setup-task/commit/01a4adf9db2d14c1de7a560f09170b6e0df736aa"><code>01a4adf</code></a>
chore: release v2.1.0</li>
<li><a
href="https://github.com/go-task/setup-task/commit/56fc0886350e15a75ed1e6f4b7a83d3b831b3054"><code>56fc088</code></a>
fix(taskfile): make mktemp utilities portable across BSD and GNU</li>
<li><a
href="https://github.com/go-task/setup-task/commit/4de50203767624993e228e2135c1d13cc156b095"><code>4de5020</code></a>
chore(release): add release task automation (<a
href="https://redirect.github.com/go-task/setup-task/issues/14">#14</a>)</li>
<li><a
href="https://github.com/go-task/setup-task/commit/f95f6c5aebc71143d70361ab79d87c447fd4c48f"><code>f95f6c5</code></a>
chore(deps): update all dependencies (<a
href="https://redirect.github.com/go-task/setup-task/issues/12">#12</a>)</li>
<li><a
href="https://github.com/go-task/setup-task/commit/dc4f00abd355059e622d428a1a905dfcd1169477"><code>dc4f00a</code></a>
chore(deps): update all dependencies (<a
href="https://redirect.github.com/go-task/setup-task/issues/2">#2</a>)</li>
<li><a
href="https://github.com/go-task/setup-task/commit/035a5f11fa6bbb298cc436d4173402c6ebfbc411"><code>035a5f1</code></a>
chore: modernize stack (<a
href="https://redirect.github.com/go-task/setup-task/issues/5">#5</a>)</li>
<li><a
href="https://github.com/go-task/setup-task/commit/099972a06751959896ae32ae844ed17001f59da5"><code>099972a</code></a>
docs: mark v2.0.0 release in changelog</li>
<li>See full diff in <a
href="https://github.com/go-task/setup-task/compare/3be4020d41929789a01026e0e427a4321ce0ad44...01a4adf9db2d14c1de7a560f09170b6e0df736aa">compare
view</a></li>
</ul>
</details>
<br />

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
2026-06-15 22:04:16 +00:00
dependabot[bot]andAnthony Stirling 5b20257dea build(deps): bump actions/stale from 10.2.0 to 10.3.0 (#6487)
Bumps [actions/stale](https://github.com/actions/stale) from 10.2.0 to
10.3.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/actions/stale/releases">actions/stale's
releases</a>.</em></p>
<blockquote>
<h2>v10.3.0</h2>
<h2>What's Changed</h2>
<h3>Bug Fix</h3>
<ul>
<li>Enhancement: ignore stale labeling events by <a
href="https://github.com/shamoon"><code>@​shamoon</code></a> in <a
href="https://redirect.github.com/actions/stale/pull/1311">actions/stale#1311</a></li>
</ul>
<h3>Dependency Updates</h3>
<ul>
<li>Upgrade dependencies (<code>@​actions/core</code>,
<code>@​octokit/plugin-retry</code>, <a
href="https://github.com/typescript-eslint"><code>@​typescript-eslint</code></a>)
by <a href="https://github.com/Copilot"><code>@​Copilot</code></a> in <a
href="https://redirect.github.com/actions/stale/pull/1335">actions/stale#1335</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/shamoon"><code>@​shamoon</code></a> made
their first contribution in <a
href="https://redirect.github.com/actions/stale/pull/1311">actions/stale#1311</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/stale/compare/v10...v10.3.0">https://github.com/actions/stale/compare/v10...v10.3.0</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/actions/stale/commit/eb5cf3af3ac0a1aa4c9c45633dd1ae542a27a899"><code>eb5cf3a</code></a>
chore: upgrade dependencies and bump version to 10.3.0 (<a
href="https://redirect.github.com/actions/stale/issues/1335">#1335</a>)</li>
<li><a
href="https://github.com/actions/stale/commit/db5d06a4c82d5e94513c09c406638111df61f63e"><code>db5d06a</code></a>
Enhancement: ignore stale labeling events (<a
href="https://redirect.github.com/actions/stale/issues/1311">#1311</a>)</li>
<li>See full diff in <a
href="https://github.com/actions/stale/compare/b5d41d4e1d5dceea10e7104786b73624c18a190f...eb5cf3af3ac0a1aa4c9c45633dd1ae542a27a899">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=actions/stale&package-manager=github_actions&previous-version=10.2.0&new-version=10.3.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
2026-06-15 22:04:09 +00:00
dependabot[bot]andAnthony Stirling 2f5fc7be4e build(deps): bump depot/build-push-action from 1.17.0 to 1.18.0 (#6488)
Bumps
[depot/build-push-action](https://github.com/depot/build-push-action)
from 1.17.0 to 1.18.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/depot/build-push-action/releases">depot/build-push-action's
releases</a>.</em></p>
<blockquote>
<h2>v1.18.0</h2>
<h2>What's Changed</h2>
<ul>
<li>Upgrade action runtime to Node 24 (<a
href="https://redirect.github.com/depot/build-push-action/issues/48">#48</a>)
<a href="https://github.com/Akatama"><code>@​Akatama</code></a></li>
<li>Add Depot Registry save example (<a
href="https://redirect.github.com/depot/build-push-action/issues/47">#47</a>)
<a href="https://github.com/maschwenk"><code>@​maschwenk</code></a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/depot/build-push-action/commit/98e78adca7817480b8185f474a400b451d74e287"><code>98e78ad</code></a>
Merge pull request <a
href="https://redirect.github.com/depot/build-push-action/issues/48">#48</a>
from depot/upgrade-node-24-runtime</li>
<li><a
href="https://github.com/depot/build-push-action/commit/e97ebff18729ac91be067461138674a006ab9bff"><code>e97ebff</code></a>
Remove Node 24 compatibility docs</li>
<li><a
href="https://github.com/depot/build-push-action/commit/2db929fa768ebb3ad332ae8fc530412bf0964782"><code>2db929f</code></a>
Upgrade action runtime to Node 24</li>
<li><a
href="https://github.com/depot/build-push-action/commit/f78af826a1c272c4b60c485e934974b515094928"><code>f78af82</code></a>
Merge pull request <a
href="https://redirect.github.com/depot/build-push-action/issues/47">#47</a>
from maschwenk/maschwenk/add-depot-registry-example</li>
<li><a
href="https://github.com/depot/build-push-action/commit/6855818d5954fa4361879bb0b0e5c32856fc6703"><code>6855818</code></a>
Update action.yml</li>
<li><a
href="https://github.com/depot/build-push-action/commit/b984f6a1944d5420eefb2b012d6eb856249bd225"><code>b984f6a</code></a>
Clarify save/save-tag/save-tags input descriptions</li>
<li><a
href="https://github.com/depot/build-push-action/commit/1a34abd3707433f4f7b6d594e49e56b4b9f4d6d0"><code>1a34abd</code></a>
Add Depot Registry save example</li>
<li>See full diff in <a
href="https://github.com/depot/build-push-action/compare/5f3b3c2e5a00f0093de47f657aeaefcedff27d18...98e78adca7817480b8185f474a400b451d74e287">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=depot/build-push-action&package-manager=github_actions&previous-version=1.17.0&new-version=1.18.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
2026-06-15 22:04:03 +00:00
Anthony Stirling d18caf6116 Fix PDF text selection locked out on touch devices (#6656) 2026-06-15 23:04:43 +01:00
dependabot[bot]andAnthony Stirling 3bafbb1919 build(deps): bump docker/metadata-action from 6.0.0 to 6.1.0 (#6490)
Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
Signed-off-by: dependabot[bot] <support@github.com>
2026-06-15 23:03:55 +01:00
James Brunton 42c1cce56d Fix Java formatting 2026-06-15 15:05:26 +01:00
fb6a118be9 Update SaaS to latest main (#6667)
# Description of Changes
> [!warning]
> **Do not** squash this on merge. It should be merged via a merge
commit

Fixes conflicts in `pgvector_store.py`. 

Also since codespell is failing, add comments to ignore the errors in
`sync_en_us_spelling.py`

---------

Co-authored-by: Ludy <Ludy87@users.noreply.github.com>
Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
2026-06-15 14:53:22 +01:00
James Brunton c55beacead Shut codespell up 2026-06-15 14:20:25 +01:00
James Brunton 04d68c650a Merge remote-tracking branch 'origin/main' into saas-update
# Conflicts:
#	engine/src/stirling/documents/pgvector_store.py
2026-06-15 14:10:21 +01:00
Anthony Stirling 9d7467cf90 Scope signing user picker to team for multi-tenant SaaS (#6583) 2026-06-15 13:44:34 +01:00
James Brunton 2a905c01c3 SaaS tidying (#6665)
# Description of Changes
* Remove complex port selection logic from `engine.yml`. It's
inconsistent with the frontend & backend task files, and caused issues
with Docker, which have been worked around but would be simpler to just
get rid of the problem altogether
* Fix Ruff formatting of Python script
* Remove payg tests which are failing and have drifted too far from the
implementation to save directly
2026-06-15 13:21:33 +01:00
Anthony Stirling d6a5777c69 Fix pgvector (#6591) 2026-06-15 13:09:40 +01:00
James Brunton c1a637d764 Fix CI errors in SaaS (#6662)
# Description of Changes
Fix CI errors in #6578 to make SaaS branch ready for merge into main
2026-06-15 11:26:29 +01:00
LudyandJames Brunton 085ad6c784 build(taskfile): use platform-specific Gradle wrapper command (#6355)
Co-authored-by: James Brunton <jbrunton96@gmail.com>
2026-06-14 15:50:31 +01:00
Ludy cb13102117 chore(frontend): make Vite allowed hosts configurable (#6354) 2026-06-14 15:49:45 +01:00
Ludy 1fa1293b39 chore(build): upgrade Gradle wrapper and Docker build images to 9.5.1 (#6501) 2026-06-14 15:48:44 +01:00
Ludy 1ce765ab1e refactor: replace legacy Paths usage with Path.of (#6441) 2026-06-14 15:48:28 +01:00
Ludy dad2425c27 docs: Document pluralization suffix usage for translations (#6650) 2026-06-14 15:47:57 +01:00
Anthony Stirling eefa8eff61 Route mobile scanner API and vendor loads through the app base path (#6648) 2026-06-12 16:00:11 +01:00
Reece Browne 63ecbe3b6d Policies: centre the collapsed-rail policy button between its dividers (#6646) 2026-06-12 13:39:03 +01:00
Anthony Stirling f1ed850a73 Fix SaaS mobile scanner being auth-gated under /app base path (#6642) 2026-06-12 13:13:49 +01:00
Anthony Stirling b11c272e87 Feature/v2/guest action gating (#6643) 2026-06-12 13:13:38 +01:00
Reece Browne f5e697347b Policies: drop the Pro-license gate from the policy API (#6645)
`PolicyController` was annotated `@PremiumEndpoint` (requires a
Pro-or-higher server license). Policies don't need a server-license
gate:

- On SaaS the server runs in Pro mode, so the check is always satisfied
anyway — it gates nothing in practice.
- Access is already governed by team scoping (#6632) plus the per-user /
guest gates.

So the annotation is dead weight and misleading. This removes it (and
its import) — a 2-line change.

## Verification
- `:proprietary:compileJava` succeeds; spotless clean. No other premium
gate on policy classes.
2026-06-12 13:13:01 +01:00
Reece Browne 4e880c7510 Policies: summon the guest sign-up banner when a guest clicks a policy (#6644)
Guests (anonymous users on a login-enabled deployment) could open a
policy's setup/detail. Policies are an account feature, so a guest
clicking a policy should be nudged to sign up rather than opening it.

## Behaviour
A guest clicking a policy row — or a collapsed-rail icon — now
**re-summons the existing guest sign-up banner** ("You're using Stirling
PDF as a guest!…") and does **not** open the policy.

- `GuestUserBanner` listens for a `stirling:show-guest-banner` window
event and re-shows (even if previously dismissed; the render guard still
hides it for non-anonymous users).
- The policy sidebar dispatches that event on a guest click (cross-layer
via `CustomEvent`, same pattern as `payg:signupRequired`; a no-op on
builds without the banner).
- `usePolicyGuestBlocked()` gates it: `config.enableLogin === true &&
user.is_anonymous === true`.
- **Login-disabled single-user** deployments have an anonymous local
operator with full access → not gated.

## Verification
- Typecheck clean (proprietary + saas); eslint clean; sidebar tests
pass.

## Note
No dedicated guest unit test — the suite mocks `useAppConfig` at module
scope, and making it per-test controllable needs `vi.hoisted` plumbing
that risked the existing tests. Easy follow-up.
2026-06-12 13:10:37 +01:00
James Brunton 511b92b321 De-AI the onboarding prose (#6641)
# Description of Changes
De-AI the onboarding prose.
2026-06-12 11:39:19 +01:00
ConnorYoh 87723d3ce2 fix(payg): fire the usage-limit modal when an AI agent run hits the limit (#6638)
## Problem

We're getting 402s when an AI **agent** (chat) run hits the free
allowance / spending cap, but the frontend handles them poorly and never
pops the usage-limit modal.

The agent runs its tool calls **server-side** (loopback HTTP via
`PolicyExecutor`), so the 402 never reaches the `apiClient` interceptor
that pops the modal for direct calls. It was caught by the generic
tool-failure handler and flattened into a `CANNOT_CONTINUE` reason
string (`"The /api/v1/… tool failed: 402…"`), streamed as a `result`
event, and rendered as a scary chat bubble. This is the same gap the
policy auto-run path bridges (#6626) — one layer up.

## Fix

**Backend** (`proprietary`)
- `AiWorkflowResponse` gains `errorCode` + `errorSubscribed`.
- `AiWorkflowService` detects a downstream 401/402 entitlement sentinel
in its three tool-exec catch sites (`onToolCall`, `runPlan`,
`onConvertMarkdown`) and surfaces the structured code (+ `subscribed`)
on the terminal response instead of the raw failure text.
- Factored the 401/402 body extraction `PolicyEngine` already had into a
shared `DownstreamEntitlementError` util so the two server-side paths
can't drift.

**Frontend**
- New `usageLimitBridge` (`PAYG_LIMIT_REACHED_EVENT` +
`dispatchPaygLimitReached`) generalises the previously policy-only
bridge. Proprietary can't import the saas modal API (layering), so
server-side limit hits broadcast a window event the saas
`UsageLimitModalHost` opens the modal from. Migrated the policy path
onto it.
- `ChatContext` fires the matching modal (free → subscribe, subscribed →
raise cap) on the limit result **and** on a direct 402, replacing the
raw reason with a brief friendly line
(`chat.responses.usage_limit_reached`).

No Python engine changes — the charge/402 happens on the Java tool
endpoint that Java itself calls.

## Test plan

- [x] `:proprietary:compileJava` + `spotlessCheck` clean
- [x] `AiWorkflowServiceTest` + `PolicyEngineTest` green
- [x] eslint, proprietary + saas typechecks clean
- [ ] Manual: drive an agent run over the limit → brief line in chat +
the right modal (free vs cap)

> Note: proprietary test compilation is currently blocked on the
pre-existing `InitialSecuritySetupTest` 6-arg ctor break (unrelated,
tracked separately); verified locally by temporarily patching it.
2026-06-12 11:38:07 +01:00
EthanHealy01 eb2527fc7f Properly sync US and GB translation files (#6635)
add en-US changes to SaaS, previously merged into main. So this is
effectively a main -> SaaS PR also. It seems to be all additive.

Also take the 230 ish missing translations from en-GB over to en-US
using a script, and also make and english spellings American when adding
them to the en-US file, and fix any existing American spellings in the
en-GB file.
2026-06-12 11:18:25 +01:00
James Brunton d363a1e957 Improve search logic (#6637)
# Description of Changes
Search has got significantly worse since #6581, where I added all the
missing tags for tools that should have been there for months. Turns out
that the fuzzy matching search logic has always been way too permissive
to match words with Levenshtein distances way too far away from the
target word, so long searches include way too much stuff. The new tags
just exposed that underlying logic issue. This PR makes the Levenshtein
logic much stricter, so it is still tolerant to minor typos in tool
names, but doesn't match completely inappropriate strings.
2026-06-12 10:34:11 +01:00
James Brunton ea102cdb93 Merge pull request #6636 from Stirling-Tools/SaaS-update
Update SaaS to latest main
2026-06-12 10:19:35 +01:00
James Brunton d995471a55 Merge remote-tracking branch 'origin/main' into SaaS-update
# Conflicts:
#	frontend/editor/src/proprietary/components/chat/ChatContext.tsx
#	frontend/editor/src/saas/components/shared/TrialStatusBanner.tsx
2026-06-12 09:58:40 +01:00
Reece Browne e3e49c07ae Policies: let team leaders configure policies in the UI (#6634)
Frontend follow-up to #6632 (team-scoped policies, editing gated to team
leaders on the backend). Brings the UI's edit gate in line.

## Problem
The policy config UI gated editing to `config.isAdmin`. On SaaS, org
users are never the single global admin, so **no one could open the
policy editor** — the same lockout #6632 fixed on the backend.

## Fix
`usePolicies` now allows a **team leader** to configure, falling back to
a global admin self-hosted:

```ts
canConfigure =
  config != null && (!config.enableLogin || isTeamLeader || config.isAdmin === true);
```

- SaaS → `isTeamLeader` (from `useSaaSTeam()`) — team leaders can
configure; members get the read-only surface.
- Self-hosted → `config.isAdmin` (the core `useSaaSTeam` stub returns
`false`, so admins aren't locked out).
- Login disabled (single-user) → always allowed.
- The `config != null` guard keeps the gate closed until app-config
resolves, so edit controls never flash for users who can't use them.

The two locked-policy banners now read "Contact a team leader to change
this policy" (updated in the `t()` defaults and the `en-GB`
translations).

## Verification
- Typecheck clean (proprietary + saas); eslint clean.
- Tests pass: `usePolicies`, `PoliciesSidebar`.
2026-06-11 23:51:26 +01:00
EthanHealy01 962119e14f UI ux/add ai warning and change style (#6633)
<img width="394" height="426" alt="Screenshot 2026-06-11 at 11 39 50 PM"
src="https://github.com/user-attachments/assets/15805931-73fd-416b-841b-99a556468433"
/>
bottom text input is sticky even when shrunk down
2026-06-11 23:49:52 +01:00
Reece BrowneandClaude Opus 4.8 cc1235bbf2 i18n(policies): route policy UI strings through i18n (English only) (#6628)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 23:37:43 +01:00
Reece Browne e88d22d2fc Policies: scope to the owning team; editing restricted to team leaders (#6632) 2026-06-11 23:21:48 +01:00
Anthony Stirling ddf10f0aaf Stop advertising mcp.tools scopes in OAuth metadata when scope enforcement is disabled 2026-06-11 23:03:43 +01:00
EthanHealy01 b756b5befb add agent warning and update style (#6629) 2026-06-11 21:55:51 +01:00
ConnorYoh eddc54c6c0 fix(payg): land usage-limit modal CTAs on the Plan section (#6630) 2026-06-11 21:42:59 +01:00
ConnorYoh 22379fd5ab fe(payg): show the usage-limit modal when the limit is hit (direct + policy) (#6626) 2026-06-11 21:28:44 +01:00
Reece Browne 6f1c19c179 Policies: enforce input on uploads only; badge follows edited files (#6627) 2026-06-11 21:27:44 +01:00
Reece BrowneandClaude Opus 4.8 ef65e6b015 feat(policies): org-wide policies with admin-only editing (#6625)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 21:23:06 +01:00
Anthony Stirling 47e5977a31 Drop startup credit-reset catch-up (bulk per-user loop; lazy reset covers it) 2026-06-11 21:21:28 +01:00
Anthony Stirling 3a4b340313 Remove legacy CreditBackfillRunner (PAYG replaces it; rows created lazily) 2026-06-11 21:03:09 +01:00
EthanHealy01 41d2aa8174 UI ux/move footer links to settings (#6606)
<img width="2056" height="1044" alt="Screenshot 2026-06-11 at 2 15
34 PM"
src="https://github.com/user-attachments/assets/e58a9f8f-7172-4f30-ab28-0760b66249c9"
/>
<img width="2056" height="1045" alt="Screenshot 2026-06-11 at 2 15
43 PM"
src="https://github.com/user-attachments/assets/890b7a0b-740f-4c7f-9a48-c9a2c28e8ded"
/>
2026-06-11 20:43:33 +01:00
ConnorYoh ee9fdeed6b fix(payg): run the entitlement guard before the charge interceptor (#6622)
## Problem

When the `EntitlementGuard` refuses a request with **402** (team is over
its free allowance / spending cap, or has no subscription to bill), the
handler never runs — so it must not charge. But it did: the guard (order
**1100**) ran *after* the charge interceptor (**1000**), so
`openProcess` had already written the charge before the 402, and
`afterCompletion` then billed it as "customer paid for the attempt" — a
ledger debit, and a **Stripe meter for a subscribed-over-cap team**.

## Fix

**Run the guard first** (order **900**, before the charge interceptor at
1000). Spring runs interceptors in ascending order on the way in and
**skips a later interceptor's `preHandle` (and `afterCompletion`)
entirely once an earlier one returns `false`** — so a refused request
short-circuits with its 402 *before* the charge interceptor runs at all.
A blocked request never opens a process, materialises inputs, or writes
a charge.

This replaces the earlier attribute-flag + `afterCompletion`-refund
approach with a simpler reorder (per review): the no-charge-on-block
guarantee is now **structural**, and it also avoids the wasted
open-then-refund churn (no temp-file write, no debit/refund pair) for
refused requests.

### Why the reorder is safe
- `EntitlementGuard` reads no `PaygChargeInterceptor` state and has **no
`afterCompletion`** (only `preHandle`), so reverse-order teardown is a
non-issue.
- The legacy `UnifiedCreditInterceptor` (default order **0**, and only
registered under the `legacy-credits` profile) still runs first, so any
legacy rejection wins.
- For *admitted* requests both interceptors still run (guard then
charge) — behaviour is unchanged; only refused requests now
short-circuit before the charge.

## Tests

`PaygWebMvcConfigTest` locks the `ENTITLEMENT_GUARD_ORDER <
INTERCEPTOR_ORDER` invariant (if it's ever reversed, refused requests
would bill again — this fails first). Existing `EntitlementGuardTest`
already proves the guard returns 402 on a degraded/billable request.
`:saas:test` + spotless green.

## Related (separate, in progress)

The **fail-cleanly + fire-the-modal** half (suppress the error toast,
trigger the subscribe/raise-cap modal via the existing
`subscribed`/`category` signal — catching the 402 centrally so direct
API usage is handled, and propagating the entitlement reason through the
async policy-run status) lands **with Ethan's modal** so we don't remove
the toast before there's a popup to replace it.
2026-06-11 20:42:40 +01:00
Anthony Stirling b1a960a240 Skip self-host user-table bootstrap (team backfill, grandfathering) in SaaS 2026-06-11 20:39:48 +01:00
Anthony Stirling 946c032fb5 Change default language to en-US and add US language (#6621) 2026-06-11 20:36:23 +01:00
EthanHealy01 7e493226c4 add popups for free limit hit and spend cap hit (#6623) 2026-06-11 20:34:59 +01:00
Anthony Stirling d48017a5b5 Skip engine free-port probe in containers (fixed port) 2026-06-11 20:18:54 +01:00
ConnorYoh 37b4d24a95 fix(payg): gate + charge AI document tools and AI Create sessions (#6617)
## Problem

Two AI surfaces slipped through PAYG unbilled:

1. **AI document tools** — `/api/v1/ai/tools/**`
(`PdfCommentAgentController`, `MathAuditorAgentController`) live in the
**proprietary** module, which can't depend on `saas` and so can't carry
the saas-only `@RequiresFeature`. They also lacked
`@AutoJobPostMapping`, so the charge interceptor's scope gate
short-circuited them **before** category resolution: **not charged, and
not even entitlement-gated** — whether called directly or dispatched by
the orchestrator.
2. **AI Create** — `/api/v1/ai/create` is JSON/session-based with no
file input, so the multipart charge path never fired. The old
per-generation charge ran through the now-dead legacy credit system, so
it currently charges nothing.

## Fix

- **`AiToolRoutes`** (new, saas) — single source of truth for the
`/api/v1/ai/tools/**` prefix. The proprietary controllers stay
untouched; the saas hot-path recognises them by path:
- **`PaygChargeInterceptor`**: brings these routes into scope and bills
them **AI** on a direct call. An orchestrator-dispatched call still
resolves to **AUTOMATION** first (the `X-Stirling-Automation` header is
checked before the path rule), so AI-tool-inside-a-workflow keeps
billing as automation.
  - **`EntitlementGuard`**: gates them on **`AI_SUPPORT`**.
- This keeps the `proprietary → saas` layering intact (no backwards
dependency).
- **`JobChargeService.chargeStandalone(ctx, units)`** — charges a fixed
unit count for a non-file billable action, reusing the existing
free-grant split + shadow row + ledger debit + `close()`→meter path on a
standalone bookkeeping job (no lineage inputs, so nothing lineage-joins
it). **`JobService.open(ctx, docUnits)`** opens that bare job.
- **`AiCreateController.createSession`** — charges **one document per
session** at creation (best-effort; entitlement is already enforced
upstream by the class-level `@RequiresFeature(AI_SUPPORT)`). Follow-up
edits (`outline` / `reprompt` / `draft` / `template` / `stream`) carry
**no** charge — they have no charge hook, so "charge on create,
follow-ups free" falls out naturally.

Per the agreed scope: charge AI Create on create now; we can optimise
follow-up handling later. **AI workflow categorisation (AUTOMATION vs
AI) intentionally left as-is** (the orchestrator's automation header
dominates by design).

## Tests

- `PaygChargeInterceptorTest`: AI-tool route (no annotations) is in
scope + **AI** category; same route with the automation header →
**AUTOMATION**; a plain non-AI route still short-circuits.
- `EntitlementGuardTest`: AI-tool route is in scope + gated on
**AI_SUPPORT** (degraded team → 402; anonymous → 401 with `category:
AI`).
- `JobChargeServiceTest`: `chargeStandalone` charges + meters the paid
portion for a subscribed team, draws the free grant (no meter) for an
unsubscribed team, and rejects `BYPASSED`.

`:saas:test` + `:saas:spotlessCheck` green; coverage gates met.

## Follow-ups (not in this PR)

- Make AI Create follow-ups explicitly cheaper / chained if we want
(currently free by absence of a hook).
- Decide whether AI-tool-inside-a-workflow should bill as AI rather than
AUTOMATION.
2026-06-11 20:10:44 +01:00
ConnorYoh aaa2599e23 fix(saas): block accepting an invite when it would orphan a paid team (#6616)
## Problem

A **free team can invite a paid team's leader** to join. The leader
accepts, and:
- `acceptInvitation` moves them onto the inviting team and removes their
membership from the old team, but only deletes the old team if it's
**personal**.
- Their paid (non-personal) team is left **memberless but still
subscribed** — an orphaned Stripe subscription billing for a team nobody
is in.

## Root cause

Two gaps in `SaasTeamService.acceptInvitation`:

1. The pre-accept guard checks `hasPaidSubscription(acceptingUser)` →
`existsActivePaidSubscriptionForUser(supabaseId)`, keyed on
**`user_id`**. A team's plan is keyed on **`team_id`**
(`existsActiveSubscriptionForTeam`), so a paid team's *leader* isn't
caught and accepts freely.
2. The 'leave existing teams' loop deletes the membership directly and
only marks **personal** teams for deletion — it bypasses the last-leader
protection `leaveTeam` already enforces (`"Cannot leave as the last team
leader. Transfer leadership first."`), and never cleans up / cancels the
non-personal team.

There is no in-app subscription-cancel path (cancellation is
Stripe-portal/webhook driven), so nothing reconciles the orphan after
the fact — it has to be prevented.

## Fix

Add `assertCanLeaveCurrentTeamsToJoinAnother(user)`, called in
`acceptInvitation` before any membership changes. For each non-personal
team where the user is the **last leader**, the accept is rejected:
- team has an active subscription → *"Cancel the plan or transfer
leadership before joining another team."*
- otherwise → *"Transfer leadership before joining another team."*

This mirrors the protection `leaveTeam` already has and is
team/leadership-aware, closing the `user_id`-vs-`team_id` gap. Regular
members and teams with another leader are unaffected.

## Verification

- `ENABLE_SAAS=true ./gradlew :saas:compileJava` — passes.
- Manual: with a paid team leader, accepting an invite to another team
should now be rejected with the message above; verify a non-leader
member can still accept.

## Note

This prevents *new* orphans. Any teams already orphaned by this bug
(memberless, still subscribed) would need a one-off reconciliation —
happy to follow up with a query/cleanup if useful.
2026-06-11 20:10:21 +01:00
EthanHealy01 33026e1a82 update saas onboarding (#6619)
<img width="1002" height="487" alt="Screenshot 2026-06-11 at 6 20 10 PM"
src="https://github.com/user-attachments/assets/5ee3cfc2-6c4f-4b35-9586-ef45fa216c6a"
/>
2026-06-11 19:39:02 +01:00
Anthony Stirling 1d598d5caa MCP OAuth discovery fix + Supabase consent page (#6608) 2026-06-11 18:30:49 +01:00
ConnorYoh 9e5fe2f4ca fix(payg): attribute policy runs to the owner so usage is charged (#6620)
## Problem

A policy ran over a file but the owner's **free usage was never
consumed**.

A policy run executes on a **background virtual thread**
(`PolicyEngine.submit` → `asyncExecutor`), and Spring's
`SecurityContextHolder` is thread-local — so the worker thread has no
identity. When `PolicyExecutor` → `InternalApiClient.post` resolves the
tool-call API key via `UserService.getCurrentUsername()`, it finds
nothing and falls back to the **`INTERNAL_API_USER`** key. The loopback
tool calls then authenticate as that system account, so
`PaygChargeInterceptor` attributes the charge to *its* team (or none) —
the real owner's free grant is untouched. Folder-watch / scheduled
triggers are even further removed (fired from a background watch loop
with no request context at all).

The charging *mechanism* was fine (AUTOMATION, multipart,
`openProcess`); only the **attribution** was wrong.

## Fix

Propagate the acting identity onto the worker thread using the
**audit-principal MDC key** that `UserService.getCurrentUsername()`
already reads as its documented async fallback (the same mechanism used
for other async jobs). No new plumbing through the executor.

- **`runPolicy`** (stored policies — covers triggers *and* manual
`runWith`) → bill the **policy owner**. `Policy.owner` is the username
stamped at creation, so `getApiKeyForUser(owner)` resolves it.
- **`submit`** (ad-hoc Automate/AI one-offs) → bill the **submitting
user**, captured on the request thread (it doesn't survive the hop to
the worker otherwise).

With the principal set, `InternalApiClient` dispatches each tool call as
that user → the interceptor resolves the right team → free grant draws /
Stripe meters correctly.

## Tests

`PolicyEngineTest`:
- `runPolicyDispatchesToolCallsAsTheOwner` — asserts MDC
`auditPrincipal` == the policy owner at the moment
`InternalApiClient.post` is invoked.
- `adHocRunDispatchesToolCallsAsTheSubmittingUser` — asserts it's the
submitting user for an ad-hoc run.

`:proprietary:test` + `:saas:test` + spotless green; coverage gates met.

## Heads-up (not in this PR)

Once attributed, **automatic folder-watch / scheduled runs consume free
grant (or bill) per file** — set up once, runs forever. That's
automation-is-billable working as intended, but a set-and-forget policy
can drain an allowance fast, so it may warrant a per-policy cap or a
heads-up in the UI. Flagging for a product decision.
2026-06-11 18:28:58 +01:00
Reece Browne 9ee0bc4b32 Policies: enforce on upload or export (#6614)
Follow-up to #6604 (merged). Builds the Security policy out so it
actually enforces, driven from the editor.

## What it does
- **Run on upload or export** — a single choice in the wizard: enforce
when a file is uploaded, or just before it's exported.
- **Output** — enforced result is a **new version** of the file
(default) or a **new file**, with optional filename
prefix/suffix/auto-number ("Output filename" subsection; auto-number
only for new files).
- **Export enforcement** — exporting an export-mode file runs the policy
first and downloads the enforced result; never hard-blocks (on failure
the original downloads). For new-version policies the in-editor file is
versioned too. Covers every export path incl. multi-file ZIP. A toast
(glowing in the policy's accent while it runs) reports progress and
fades after ~10s.
- **Affordances** — a freshly enforced file briefly glows its policy
accent and carries a shield badge.
- **Config tidy-up** — removed the unwired Security setting fields + the
wizard's review step; "Upgrade to enterprise" on locked categories;
category accent in the detail/wizard headers.

## Notes
- Builds on the manual-only (client-driven) policy model from #6587
(`trigger: null`, metadata in `output.options`), adding the `runOn`
field + export-time enforcement.
- The page-editor merge-export (no single source file) enforces +
downloads but doesn't version in place.

## Verification
typecheck (core + proprietary), eslint, prettier; proprietary suite
(105) green; flows checked in-app.
2026-06-11 18:12:01 +01:00
ConnorYoh 5fa5e12c64 fix(saas): show team invitation banner in SaaS web build (#6612)
## Problem

When a user is invited to a team, the SaaS web app shows **no invitation
banner** — even though the pending invite is returned by
`/api/v1/team/invitations/pending` on refresh.

## Root causes

1. **Never rendered in SaaS.** `TeamInvitationBanner` only existed in
`desktop/`, wired solely into `DesktopBannerInitializer`. The SaaS
banner stack rendered only `<UpgradeBanner />`.
2. **Single banner slot.** `BannerContext` holds one node; `setBanner`
replaces it. `TrialStatusBanner` called `setBanner(null)` when there was
no active trial (and re-fired once `trialStatus` resolved async), wiping
any other banner.
3. **Shadowing was too fragile.** A first attempt shadowed the
proprietary `UpgradeBannerInitializer` from the saas layer, but
`vite-tsconfig-paths` resolves the `@app` specifier once at dev-server
start — a newly-added shadow of an already-resolved module isn't picked
up on a browser refresh, only a full restart. So the proprietary
initializer kept running and no invite banner appeared (while the SaaS
team context still fetched + populated the invite, which is why the
pending call was visible).

## Fix

- Add `saas/components/shared/TeamInvitationBanner.tsx` — ported from
desktop, minus the desktop `connectionMode` gate and explicit billing
refresh (SaaS `acceptInvitation` already refreshes credits + session).
- Render it **inline in `saas/routes/Landing.tsx`** next to
`GuestUserBanner` — a new import specifier in an existing file
(HMR-friendly), unambiguously inside `SaaSTeamProvider`, mirroring the
proven `GuestUserBanner` pattern. No dependency on the single banner
slot.
- **Remove `TrialStatusBanner`** (trials are being retired) so it can't
clobber banners. Also drops the stale mention from the stripe-lazy-load
test comment.

## Verification

- `tsc --noEmit -p tsconfig.saas.vite.json`: clean in touched files;
total unchanged from baseline (37 pre-existing, unrelated).
- Manual: pull + verify the Accept/Decline banner appears for an account
with a pending invite.
2026-06-11 18:01:58 +01:00
James Brunton 34ead60194 Kill off agents pane now that we have FAB (#6613)
# Description of Changes
Kill off agents pane now that we have the FAB. Also fixes a bug with the
FAB where it would sometimes fail to render the chat, and fixes a
duplicated entry in the Vite config which was throwing a warning
2026-06-11 17:28:05 +01:00
ConnorYoh 5bc7ae626d fix(payg): cancelled subscription left team gated as subscribed (#6611)
## Problem

A team that **cancelled** its PAYG subscription kept full subscribed
access:

- **UI didn't reflect cancellation** — the Plan tab still rendered the
subscribed view, never the free/upgrade view.
- **Automation wasn't stopped** — automation / AI / API kept running
without ever falling back to the free-grant gate.

## Root cause

`TeamBillingService.compute` decided `subscribed` as:

```java
boolean subscribed =
    subscriptionId != null
        || extOpt.map(PaygTeamExtensions::getStripeCustomerId).filter(s -> !s.isBlank()).isPresent();
```

On cancellation, the `customer.subscription.deleted` webhook calls
`payg_unlink_subscription`, which nulls `payg_subscription_id` but
**deliberately keeps `stripe_customer_id`** (so a future re-subscribe
can reuse the Stripe customer).

`payg_link_subscription` is the **only** writer of
`payg_team_extensions.stripe_customer_id`, and it writes it in the
*same* `UPDATE` as `payg_subscription_id` (on
`customer.subscription.created`). So the customer id is never set before
the subscription id — the "pre-webhook stand-in" the old comment claimed
**cannot happen**. The fallback only ever pinned a team that *ever*
subscribed to `subscribed` forever, because the Stripe customer outlives
the subscription.

Both symptoms are this one flag:
- `PaygWalletController` status → `SUBSCRIBED` vs `FREE`
- `EntitlementService` gate branch → monthly-cap vs free-grant

## Fix

Gate `subscribed` purely on `payg_subscription_id != null`. A cancelled
team now correctly drops to free (UI shows free; billable ops gate on
the one-time grant). This aligns the wallet/entitlement read with the
**meter path** (`JobChargeService.close`), which already gated on
`payg_subscription_id`.

Handles both Stripe cancel modes: "cancel at period end" keeps the sub
`active` (id stays set) until `.deleted` fires at period end → access
through the paid period; immediate cancel fires `.deleted` now → flips
to free now.

**No data migration / backfill** — already-cancelled teams have
`payg_subscription_id = NULL`, so they flip to free as soon as this
ships (within the 30s billing-cache TTL).

## Tests

Adds `TeamBillingServiceTest` — the `subscribed` computation previously
had **no** unit coverage (which is how this shipped). Covers: subscribed
iff subscription id present; **cancelled team (customer id remains,
subscription id null) ≠ subscribed** + free grant survives;
no-subscription/no-customer; no extension row.

`:saas:test` + `:saas:spotlessCheck` green; coverage gates met.

## Not included (optional hardening, can fast-follow)

- Cross-check the synced `stripe.subscriptions.status` to guard a
*missed* `.deleted` webhook leaving `payg_subscription_id` stale.
- Push cache-invalidation from the webhook (currently ≤30s TTL
staleness).
2026-06-11 17:26:58 +01:00
ConnorYoh f16ca4795c fe(payg): remove em dashes from Plan page copy (#6610)
## What

Removes all em dash (`—`) characters from the **user-facing text** on
the Plan page (PAYG section), replacing them with colons, commas, or
restructured punctuation so the copy reads naturally.

## Changes

- `frontend/editor/public/locales/en-GB/translation.toml` — all `payg.*`
strings (this is what actually renders on the page)
- `PaygFree.tsx` — `t()` default fallbacks + the `{" — "}` JSX
benefit-list separators (now `{": "}`)
- `Payg.tsx` — `t()` default fallback for the editor-plan body

## Notes

- The en-dash range separator (`{{start}} – {{end}}`) in the
billing-period string is intentionally **kept** — only em dashes were
targeted.
- JSDoc / code comments containing em dashes were **left unchanged**,
since they aren't rendered text on the page.
<img width="990" height="502" alt="image"
src="https://github.com/user-attachments/assets/13d89b0f-007c-4b4c-b72d-1d912f968bc7"
/>
2026-06-11 16:50:27 +01:00
James Brunton d52c7ced7c Improvements to Stirling Engine to prepare for SaaS release (#6603)
# Description of Changes
- Use pool for postgres connections
- Add ability to require user ID to be set on API calls to the engine
- Add process-wide concurrency cap on AI access (in addition to existing
user caps)
- Allow number of workers (threads) to be specified for stirling engine
- Update env var names to reflect that the DB is not just for RAG
2026-06-11 16:31:35 +01:00
James Brunton 606964ee52 Fix Teams and MCP settings pages (#6605)
# Description of Changes
Remove the Pro guards from the Team settings page and also fix the
styling of the MCP settings screen (the code sections were black text on
black background in light mode)
2026-06-11 16:26:02 +01:00
ConnorYohandReece cf513c255b PAYG: pay-as-you-go billing — metered automation/AI/API + one-time free grant (#6589)
## Summary

Pay-as-you-go (PAYG) billing for Stirling-PDF SaaS. Manual PDF editing
stays free forever; only **automation, AI, and API** usage is metered.
Every team gets a **one-time lifetime free grant** (default 500 PDFs)
before any billing; past that, a team adds a card and pays per metered
document, with a self-set monthly spending cap.

This branch combines and supersedes the in-flight BE (#6574) and FE
(#6579) work plus the SaaS edge functions (Stirling-PDF-SaaS PR, now on
`v3`), hardened into a single reviewable feature after a pre-merge
dead-code/security review.

## Billing model

- **Always free:** manual / JWT web-tool usage is `BYPASSED` — never
metered, no matter where it's triggered.
- **Billable categories:** `AUTOMATION`, `AI`, `API`.
- **One-time lifetime free grant** (`pricing_policy.free_tier_units`,
default 500): never resets, survives subscribing. It gates unsubscribed
teams (billable API calls hard-stop with a 402 once exhausted) and
decides the free-vs-paid split of every job.
- **Subscribed:** paid documents (beyond the grant) are metered to a
Stripe Billing Meter; an optional monthly spending cap degrades billable
categories when reached.
- **Dedup:** the same file pushed through several steps within a
workflow window counts **once** (lineage join), so API/AI chaining on
one file isn't double-charged.

## What's included

**Database** — Flyway migrations `V11`→`V21` with matching Supabase
twins: pricing policy + per-team sidecar (`payg_team_extensions`:
subscription id, Stripe customer, free-grant counter), append-only
`wallet_ledger`, shadow charges, subscription-state RPCs (`V14`), audit
logs (`V15`), billing category (`V16`), one-time lifetime free grant
(`V19`), launch-grant seed (`V20`), drop of the unused
`wallet_category_summary` view (`V21`).

**Charge pipeline** — `PaygChargeInterceptor` (open/join a process,
split the free grant, write the ledger DEBIT), `JobChargeService`
(consume the grant under a row lock, restore it on a first-step refund,
meter only the paid portion on completion), `StaleJobCloser` fallback
(idempotent close → meter).

**Entitlement** — `EntitlementService` (per-team cached snapshot:
grant-gated for free teams, monthly-cap-gated for subscribed) +
`EntitlementGuard` (401 `SIGNUP_REQUIRED` / 402 `FEATURE_DEGRADED` /
`PAYG_LIMIT_REACHED`).

**Metering** — `PaygMeterReportingService` writes a durable
`payg_meter_event_log` row around every POST to the `meter-payg-units`
edge fn (pending → posted/failed); `PaygMeterReconcileScheduler` retries
unposted events under the same idempotency key inside Stripe's 24h dedup
window.

**Billing facts** — `TeamBillingService` reads the synced `stripe.*`
mirror (subscription window, per-document rate; the unsubscribed-team
estimate resolves the rate by Price `lookup_key = plan:processor`).

**Wallet API** — `PaygWalletController`: `GET /api/v1/payg/wallet`,
`PATCH /api/v1/payg/cap`.

**Frontend** — PAYG Plan page (two-card free layout + subscribed views),
`useWallet`, upgrade modal with lazy-loaded Stripe Embedded Checkout and
a shared `SpendCapControl`, customer-portal link, 402/401 interceptor
toast, en-GB i18n. (Per-member usage shows each teammate's spend; the
activity feed is behind a flag until polished.)

**SaaS edge functions** (`Stirling-PDF-SaaS` `v3`) —
`create-checkout-session`, `create-payg-team-subscription`,
`create-customer-portal-session`, `meter-payg-units`,
`payg-subscription-webhook`, `stripe-sync`, plus the stripe-sync
`migrate` + scoped-`backfill` scripts. All price lookup is DB-driven (no
`STRIPE_PAYG_PRICE_ID_*` env vars).

## Release prerequisites (prod)

1. Apply Flyway migrations (`V11`→`V21`) and the Supabase migration
twins.
2. Stripe Sync Engine: run `stripe-sync:migrate`, then a **scoped**
backfill — `product`, `price`, `customer`, `subscription` only (not
`all`, which rate-limits).
3. Register 2 PAYG webhook endpoints (each its own signing secret):
`stripe-sync` (product/price/customer/subscription `.*`) and
`payg-subscription-webhook` (`customer.subscription.created`/`.deleted`
drive state; `.updated` + `invoice.*` observed). Keep the legacy
`stripe-webhook` only if credits/self-hosted flows still run.
4. Stripe Billing Meter: `event_name = payg_doc_units`, value key
`processed_documents`.
5. Env: `PAYG_METER_ENDPOINT` + `SUPABASE_EDGE_FUNCTION_SECRET`
(backend); the webhook signing secrets (edge fns). The default pricing
policy must point at the PAYG Stripe Price(s); `V20` seeds
`free_tier_units = 500`.

## Testing

- `:saas:test` green, `:saas:spotlessCheck` clean, edge-fn Deno tests
green, FE saas typecheck clean (the remaining errors are pre-existing
`proprietary/*` + `prototypes/*`, untouched here). Cucumber shadow-mode
suite + CI workflow included.

## Pre-merge review

An independent dead-code/security pass came back **clean on security**
(team-derived authz / no IDOR, leader-only cap mutation, no
billing-category downgrade, dev/mock hooks gated to
`import.meta.env.DEV` + `/dev/`, no secrets/injection, fail-open
metering by design). The dead/unwired code it flagged has been removed
in this branch (unenforced sub-cap control, an unused JDBC DAO + its
view, dead methods).

## Follow-ups (tracked, not blocking)

- **Enforce per-member sub-caps** — the control was removed because it
read for display but never gated a request; the per-member usage display
and `cap_units` column are retained for when enforcement is wired.
- **API/AI chaining billing model + `ProcessType` enum** — confirm
same-file dedup covers API chaining; define per-tool AI charging; decide
whether the unused enum values stay.
- **Activity feed** — hidden behind a flag until the meter-event surface
is polished.

---------

Co-authored-by: Reece <reece@stirlingpdf.com>
2026-06-11 15:56:01 +01:00
EthanHealy01andAnthony Stirling 88adb7adad create agent (#6520)
Added the create agent. Use [these
prompts](https://github.com/Stirling-Tools/Stirling-PDF-SaaS/blob/main/docgen/backend/default_templates/sample_prompts.md)
to test or try your own :)

Here’s the one I use

```
Hey, I need to generate an employee expense report for reimbursement.
Company: Summit Consulting Partners Company address: 88 Riverside Plaza, Suite 1400, New York, NY 10069 Accounting department email: expenses@example.com
Employee details:
* Employee Name: Michael Tran
* Employee ID: EMP-1047
* Department: Client Services
* Report Date: January 20th, 2026
* Reporting Period: January 5th, 2026 – January 16th, 2026
* Manager Approver: Laura Simmons
Trip purpose: Client onsite meetings with Atlantic Energy Solutions in Boston, MA.
Expense items:
* Flight (NYC to Boston roundtrip) — $325.40 — January 5th, 2026 — Airline ticket
* Hotel (3 nights at Harborview Hotel) — $822.75 — January 5th-8th, 2026
* Taxi from airport to hotel — $48.00 — January 5th, 2026
* Client dinner (3 attendees) — $186.20 — January 6th, 2026
* Parking at JFK Airport — $72.00 — January 5th-8th, 2026
* Breakfast (per diem not used) — $18.50 — January 7th, 2026
* Uber to client office — $22.10 — January 7th, 2026
* Printing + presentation materials — $46.90 — January 8th, 2026
* Lunch with client — $39.75 — January 8th, 2026
* Office supplies (notebooks, pens) — $27.60 — January 10th, 2026
* Mileage reimbursement (client visit in NJ, 42 miles @ $0.67/mile) — $28.14 — January 14th, 2026
* Team lunch meeting (internal) — $64.30 — January 15th, 2026
Reimbursement method should be direct deposit.
Add a notes section stating: "All receipts attached. Expenses are business-related and comply with company travel policy."
```

---------

Co-authored-by: Anthony Stirling <77850077+frooodle@users.noreply.github.com>
2026-06-11 14:18:13 +00:00
Reece Browne 11ab762f57 feat(policies): config refinements + new-version output (post-#6598) (#6604)
Follow-up to #6598 (squash-merged into `SaaS`). These are the policy
refinements made after that merge, against the current `SaaS` tip.

## Changes
- **Simplify Security config + plain-language info buttons** — Redact
config reduced to the PII field; Sanitise has no config
(JavaScript-removal only) with a non-technical info button; per-tool
info buttons reworded to match the tool-steps style.
- **Hide 'Flatten PDF pages to images' from the watermark policy
config** — new `PolicyWatermarkConfig` wrapping the watermark settings
with the flatten checkbox gated off.
- **Flatten-to-image on by default for redact + watermark** — both
normalise `convertPDFToImage: true` on mount.
- **Self-heal a stale backing folder** — `ensurePolicyFolder` recreates
a backing folder whose `folderId` no longer resolves (preferring the
backend's stored automation), instead of hanging Edit Settings on a
permanent "Loading…".
- **Version the input file on 'new version' output mode** — completed
runs whose policy output mode is `new_version` replace the input file
with a versioned child (origin tool `automate`) rather than adding a
separate file; falls back to a new file if the input is gone.
`outputMode` is plumbed through `PolicyState`, the local-cache default,
and backend reconciliation.

## Verification
- `typecheck:proprietary` + `typecheck:core` clean
- policy + hooks vitest: 17 passing
- eslint + prettier clean on all changed files
2026-06-11 14:45:22 +01:00
James Brunton 68e031ac55 Policies tidying (#6587)
# Description of Changes
* Improve typing of API (breaking change but unreleased, frontend also
updated in this PR)
* Add ownership concept to policies
* De-AI the comments
* Update the `task dev:saas` rule to spawn the engine as well
2026-06-11 13:20:01 +01:00
Anthony Stirling c722b9f6ad fix: MCP copy buttons read as proper buttons in dark mode
Subtle/gray compact buttons rendered as low-contrast floating text;
use the default variant (adaptive surface+border) idle, light teal when
copied.
2026-06-10 17:26:43 +01:00
Anthony Stirling 36c68fb69e fix: doubled base path in mobile-scanner QR URL
A configured frontendUrl/server_url already includes the subpath (e.g.
/bpp), but the code also applied withBasePath, producing /bpp/bpp/...
Append the route directly to a configured URL; reserve withBasePath for
the bare-origin fallback. Matches the ShareFileModal convention.
2026-06-10 17:21:42 +01:00
Anthony Stirling d3c359f923 reword MCP usage tip to reference the API and Automation 2026-06-10 16:38:16 +01:00
Anthony Stirling 4947ab12fd remove the 'What your assistant can do' tool-category badges from MCP section 2026-06-10 16:33:48 +01:00
Reece Browne 8dde4262ec feat(policies): backend-driven policy enforcement (frontend) (#6598)
## Summary
Adds the **Policies** feature (proprietary, behind the
`POLICIES_ENABLED` flag): backend-driven enforcement that runs a fixed
tool pipeline on documents, docked in the right tool sidebar alongside
Tools.

## Highlights
- **Policy catalog** — 5 categories; **Security** is wired (redact PII +
sanitize), the others are marked "Coming soon".
- **Backend as source of truth** — policies persist via the Policies
engine (`/api/v1/policies`), one policy per category, with a local cache
+ offline fallback.
- **Auto-run** — enabled policies run on every uploaded file: dispatch →
poll → import outputs into the workspace.
- **Security redact config** — PII preset dropdown + custom word/regex
entry + advanced options; tool params map to the backend endpoint
fields.
- **Activity feed** with retry on failures; **file badges** showing
which policies ran on a file (sidebar + files page), tinted to the
policy colour.
- Reuses the **Watched Folders** engine for each policy's backing
folder; policy-owned folders are filtered out of the Watched Folders UI.

## Notes
- Gated by `POLICIES_ENABLED` (true in proprietary, false in core) —
unreachable in the open-source build.
- Frontend-only diff; depends on the backend Policies engine and the
merged Watched Folders feature.
2026-06-10 15:57:08 +01:00
James Brunton ebc28b0a14 Add team settings to SaaS (#6601)
# Description of Changes
Add team settings UI to SaaS, which is currently only available in
desktop. It'd be nice to refactor this so they're more shared, but
they're slightly different so needs to be done with some care. Leaving
for followup work.
2026-06-10 15:54:18 +01:00
Anthony Stirling 56862cc1d3 Merge branch 'main' into SaaS 2026-06-10 15:51:43 +01:00
EthanHealy01 9b877d4f8d Move agent section to fab (#6597) 2026-06-10 15:47:47 +01:00
Reece Browne da4b84962c drop type-aware ESLint to stop the lint OOM (#6602) 2026-06-10 15:44:52 +01:00
Anthony Stirling d6306f51e1 fix: no blue disc behind the sidebar profile picture
Keep the colored background only for the initials fallback; a real
photo fills the circle with a transparent backing.
2026-06-10 15:05:29 +01:00
Anthony Stirling 9a1804ce04 Merge branch 'main' into SaaS 2026-06-10 14:58:44 +01:00
Anthony Stirling 611468b972 Add SaaS MCP usage tab (#6590) 2026-06-10 14:58:33 +01:00
EthanHealy01 5fca2f199a Feature/pdf ingestion jpdfium (#6525) 2026-06-10 14:51:41 +01:00
Anthony Stirling be0db3fd8a fix: show profile picture in the FileSidebar bottom bar
The home page's bottom-left settings button is FileSidebar's bottom
bar, which hardcoded an initials circle - the avatar work in
useConfigButtonIcon only affects the QuickAccessBar rail, which the
home page doesn't render. Add a layered useProfilePictureUrl hook
(core stub returns null; saas returns the auth context URL) and render
the picture inside the existing avatar circle, falling back to the
initial when absent or on image load failure.
2026-06-10 14:49:16 +01:00
EthanHealy01 2aa6768921 show chat progress and other UX improvements (#6576) 2026-06-10 14:47:57 +01:00
Anthony Stirling 90bda6b4b4 fix: User principal was discarded by the resource server re-authentication
The saas chain authenticates bearer requests twice:
SupabaseAuthenticationFilter builds an EnhancedJwtAuthenticationToken
with the resolved User principal, but BearerTokenAuthenticationFilter
(oauth2ResourceServer) then re-authenticates the same token through the
static toAuthentication converter and overwrites the SecurityContext
with a token whose principal is the raw Jwt - so storage endpoints kept
returning 401 "Unsupported user principal" despite the principal fix.

Carry the User across in the converter: when the context already holds
an EnhancedJwtAuthenticationToken for the same subject with a User
principal, attach that User to the converter-built token. No extra DB
lookups; anonymous sessions and API-key auth unchanged. Covered by new
unit tests (carry, no-context, subject mismatch).
2026-06-10 14:37:12 +01:00
Anthony Stirling 5b412c0fed cleanup: trim oversized comments across recent SaaS fixes
Reduce multi-paragraph comment blocks to short two-line notes and drop
history-style references; no behaviour changes.
2026-06-10 14:30:29 +01:00
Anthony Stirling bf18af4708 fix: show user avatar on the home page settings button
The bottom-left settings button and the settings page both read
profilePictureUrl, but only the settings page had a fallback (initials
avatar) - the button silently fell back to a gear. The URL itself was
usually null because fetchProfilePicture raced the background OAuth
avatar sync with a fixed 500ms delay and never retried, and a missing
bucket object simply resolved to null.

- useConfigButtonIcon: fall back to the same initials avatar as the
  settings page instead of the gear when no picture URL is available.
- UseSession: fetch the profile picture when syncOAuthAvatar settles
  (init and SIGNED_IN) instead of after an arbitrary 500ms.
- fetchProfilePicture: when the bucket copy is missing, fall back to
  the OAuth provider's own photo URL so the picture shows immediately
  on first login - unless the user explicitly uploaded/removed a
  picture (metadata source 'upload'), preserving the remove flow.
2026-06-10 14:25:19 +01:00
EthanHealy01andAnthony Stirling f15e405759 changes to the login and signup, similar to in the saas repo (#6577)
Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
2026-06-10 13:49:27 +01:00
Anthony StirlingandClaude Opus 4.8 d29059e6fb fix: storage APIs 401'd valid Supabase sessions (principal type mismatch)
FileStorageService.requireAuthenticatedUser and
FolderService.requireAuthenticatedUser authorize via
'principal instanceof User', but EnhancedJwtAuthenticationToken extends
JwtAuthenticationToken whose principal is the decoded Jwt - so every
/api/v1/storage/* request 401'd for JWT users AFTER Spring Security had
already authenticated them. This persistent 401-with-valid-session was
the trigger feeding the frontend login loop.

Attach the filter-resolved local User as the token principal for full
accounts (User implements UserDetails, matching the form-login
convention every shared instanceof check expects). Anonymous sessions
keep the raw Jwt principal, preserving their existing exclusions. All
other principal consumers verified safe: AuthenticationUtils checks
instanceof User first, extractSupabaseId/CreditController/Team
SecurityExpressions switch on the authentication type, not the
principal.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 13:44:22 +01:00
Anthony StirlingandClaude Opus 4.8 e7bbbb4702 fix: make the 401 login redirect loop structurally impossible
Audit of every code path that can produce the login->/->login cycle
found the observed loop was one instance of a repeatable class: any
automatic API call that persistently 401s while the Supabase session is
valid triggers httpErrorHandler's hard redirect to /login, which sees
the valid session and bounces back. Close the class, not just the
instance:

- saas apiClient: a 401 that survives a refresh-and-retry means the
  backend rejected a valid token (authz bug / wrong origin), not an
  expired session - never redirect to /login for it. Also fix the stale
  publicEndpoints entry ('endpoints-enabled' matched nothing; the real
  routes are endpoints-availability and endpoint-enabled).
- httpErrorHandler: sessionStorage loop breaker - if a 401 redirect
  already fired within 10s, suppress the repeat instead of cycling.
- Guard the remaining unflagged automatic callers: /api/v1/credits
  (fires on session init and TOKEN_REFRESHED), endpoints-availability
  (fires on app load), and ui-data/login (auto-called when a stale
  stirling_jwt is present).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 13:44:21 +01:00
Anthony StirlingandClaude Opus 4.8 247ef6313c fix: stop login loop caused by unguarded folder-sync 401
The deployed app looped /login -> / -> /login forever: Login sees a
valid Supabase session and navigates to /, the global FolderProvider
pulls GET /api/v1/storage/folders, the backend rejects it with 401, and
the global error handler hard-redirects back to /login?from=/bpp.

fileSyncService's /api/v1/storage/files pull already opts out via
suppressErrorToast + skipAuthRedirect, so its 401 fails silently;
folderSyncService.list() passed neither flag, so its 401 fell through to
the redirect. Add the same flags - FolderContext.pullFromServer already
handles 4xx locally (flips serverReachable, suppresses the banner).

Note: the underlying 401 on /api/v1/storage/* with a valid session is a
backend/deployment issue (storage endpoints rejecting the Supabase
token); this change makes the frontend resilient so it degrades to
"folder sync unavailable" instead of an auth loop.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 12:13:20 +01:00
Anthony StirlingandClaude Opus 4.8 7f7c865888 fix: stop unauthenticated storage calls on /login + fix subpath manifest 404
Two issues seen on the hosted /bpp login screen:

1. GET /api/v1/storage/folders fired (and 401'd) on the login page. The
   global FolderProvider pulls from the server whenever
   appConfig.storageEnabled is true, with no auth gate, so it hits the
   authenticated storage API before the user has signed in. Skip the pull
   on auth routes (/login, /signup, /auth/*, /invite, /reset-password),
   mirroring the existing LicenseContext / AppConfigContext guards. Tests
   wrap FolderProvider in MemoryRouter (now uses useLocation).

2. manifest.json and modern-logo/favicon.ico 404'd from the domain root
   instead of /bpp/. vite base for RUN_SUBPATH deploys was "/bpp" with no
   trailing slash, so <base href="/bpp"> made the browser resolve relative
   links against the parent (root). Use "/bpp/"; getBasePath() strips the
   trailing slash, so BASE_PATH, routing and asset URLs are unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 11:56:48 +01:00
James Brunton 3675db5907 Add new triggers, sources and sinks to policies (#6543)
# Description of Changes
Add new triggers:
- Schedule (fires every X amount of time)
- Folder watch (fires whenever the OS tells us a folder has a new file
in it; on Mac this is technically a 2s schedule but that's just how Java
implements it)

Add new sources:
- Folder (reads from this directory)

Add new sinks:
- Inline (stores in FileStorage)
- Folder (stores in specified directory)

Still want to do S3 buckets and web hooks and stuff, but they can come
in a future PR. I'm hoping this should make it sufficient to be able to
integrate with processing folders frontend etc. I've also changed it so
that policies can have multiple sources and triggers at once, which
seems like it might be useful.
2026-06-10 10:55:29 +00:00
Anthony Stirling 2101b4028c Merge branch 'main' into SaaS 2026-06-10 11:42:25 +01:00
Anthony StirlingandClaude Opus 4.8 06476ea69e fix(saas): collapse duplicate Supabase client to one GoTrueClient
The console warned "Multiple GoTrueClient instances detected in the same
browser context" and storage endpoints (/api/v1/storage/folders,
/files) kept 401ing even after a successful token refresh.

Cause: the SaaS bundle instantiated TWO Supabase clients on the same
sb-<ref>-auth-token storage key. :saas/auth/supabase.ts creates the
primary client (used by UseSession + apiClient), while billing /
licensing / user-management code imports @app/services/supabaseClient,
which fell through to :proprietary/services/supabaseClient.ts and called
createClient() again. Each client runs its own autoRefreshToken timer,
so they rotate the refresh token out from under each other → "Already
Used" refresh failures and spurious 401s, plus a residual /login flash.

Add a :saas override of @app/services/supabaseClient that re-exports the
single instance from @app/auth/supabase. The path mapping
(@app/* → src/saas/* → src/proprietary/* → src/core/*) now resolves
every consumer to the same client, so the :proprietary createClient() is
never bundled in the SaaS build.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 11:26:27 +01:00
Reece Browneandaikido-pr-checks[bot] e2536daeb8 Feature/v2/smartfolders rebuild (#6480)
Co-authored-by: aikido-pr-checks[bot] <169896070+aikido-pr-checks[bot]@users.noreply.github.com>
2026-06-10 11:18:14 +01:00
Anthony StirlingandClaude Opus 4.8 1135bd9b63 fix(saas): stop login→logout→login bounce on cold load
On returning to the app with an expired Supabase access token, bootstrap
requests fired with the stale token and 401'd before Supabase finished
refreshing. The global 401 handler then hard-redirected to
/login?from=… (a full window.location navigation), and once the refresh
landed the app sent the user straight back in — the login/logout/login
flicker.

Two holes in the SaaS apiClient response interceptor caused it:

1. "public" endpoints (e.g. /api/v1/config/app-config) skipped the
   refresh-and-retry path. The backend 401s any expired Bearer token
   regardless of route, so those bootstrap calls 401'd and fell through
   to handleHttpError, which redirected to /login. Now public endpoints
   also refresh-and-retry, and a 401 on a public endpoint sets
   skipAuthRedirect so it can never trigger the global login redirect.

2. Concurrent 401s each called supabase.auth.refreshSession()
   independently. Supabase rotates the refresh token on first use, so
   the racing refreshes failed with "Invalid Refresh Token: Already
   Used" and bounced the app even though the session was recoverable.
   Refreshes are now de-duplicated through a single in-flight promise.

Existing apiClient unit tests (refresh-and-retry on protected 401, bare
/login redirect on genuine refresh failure) are preserved.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 11:13:28 +01:00
Anthony Stirling bbfe29c2ef Merge remote-tracking branch 'origin/main' into SaaS
# Conflicts:
#	frontend/editor/src/core/components/shared/AppConfigModal.tsx
2026-06-10 10:51:06 +01:00
James Brunton e0fc5061de Delete dead translations (#6581)
# Description of Changes
Adds all the missing translations that I could find (they're all dynamic
ones that the existing test can't detect are required) and adds a new
test to find as many unused translations as possible. The test has an
ignore list for translations that are used, but dynamically so the test
can't find them (most of the settings UI translations are built up
dynamically like that).

This PR is scoped to just include en-GB translation changes, since
that's the main supported language. We'll need to do a translation PR to
trim all the dead keys from the other languages, and add the missing
ones.
2026-06-10 09:49:00 +00:00
Anthony Stirling 3ecd95b779 Add MCP server with OAuth/API-key auth (#6570)
Adds an optional MCP server (proprietary module) that exposes Stirling's
PDF operations and AI capabilities to MCP clients. Off by default, zero
footprint when disabled.

### What
- New `/mcp` endpoint: streamable-HTTP + JSON-RPC 2.0; 8 tools
(describe_operation, pages/convert/misc/security category tools, AI,
upload, download).
- Runs real operations over an internal loopback; results returned
inline as base64 (small) or by fileId (large).

### Auth (two modes)
- OAuth2 resource server: RFC 9728 protected-resource metadata, RFC 8707
audience binding, JWKS, `mcp.tools.read/write` scopes; binds each token
to a provisioned Stirling account.
- API-key mode: reuses Stirling per-user `X-API-KEY` (no IdP needed).

### Security
- Per-user file ownership in FileStorage: async/queued writes scoped to
the submitting user; legacy/owner-less files stay readable.
- Admin allow/block list controls which operations are exposed.
- Python engine gated behind a shared secret (`X-Engine-Auth`).
- MCP filter chain is isolated and cannot weaken the main app's
security.
- Hardened: no upstream error-body leakage, log injection sanitized,
fileId path/sidecar enumeration blocked.

### Config / footprint
- Disabled by default (`mcp.enabled=false`); all beans
`@ConditionalOnProperty`.
---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-06-10 09:46:25 +00:00
ConnorYoh 84aca12055 PR-S4: shadow-mode hardening (review follow-ups) (#6523)
## What this PR does

Bundles the **low-risk polish items** from the [multi-agent review of
#6519](https://github.com/Stirling-Tools/Stirling-PDF/pull/6519). Each
change is independent, mechanical, and ships with focused unit-test
coverage.

The medium-severity items (\`?async=true\` OUTPUT recording,
JSON-consumes endpoint coverage, SpringBootTest harness) are tracked
separately in [\`notes/PAYG_DESIGN.md\` §7.5
PR-S4](https://github.com/Stirling-Tools/Stirling-PDF/blob/payg-s4-hardening/notes/PAYG_DESIGN.md)
— they need design decisions + bigger infrastructure work, so this PR
sticks to the mechanical wins.

Stacked on #6519. When that merges to main, this rebases cleanly — no
code changes.

## Changes

| Area | What | Why |
|---|---|---|
| **\`tool_id\` becomes route pattern** |
\`PaygChargeInterceptor.resolveToolId()\` prefers
\`HandlerMapping.BEST_MATCHING_PATTERN_ATTRIBUTE\` over
\`request.getRequestURI()\`. Truncates to 128 + WARN log +
\`payg.filter.errors\` increment when truncation fires. | Audit rollups
aggregate by endpoint instead of by every individual request's
path-variable / matrix-param variant. Silent truncation now louder. |
| **Direct PDF magic-byte check** | \`PaygOutputExtractor.extract()\`
magic-checks the body even for direct \`application/pdf\` responses. |
Asymmetric with the ZIP-entry path which always magic-checks. A tool
that emits \`application/pdf\` for a JSON / HTML payload would otherwise
pollute \`job_artifact_hash\`. |
| **DESKTOP_APP detection** | \`X-Stirling-Client: desktop\` header →
\`JobSource.DESKTOP_APP\`. | The enum value was unreachable from
\`determineSource()\`; Tauri shell traffic was mis-classified as WEB. No
anti-spoof — V12 step limits are identical for WEB/DESKTOP_APP so the
worst-case abuse value is zero today. |
| **\`max-bytes\` sensible default** | 500 MiB instead of \`null\`
(unbounded). | Covers the largest realistic Stirling responses (full
split-to-ZIP on a 1000-page document) while preventing pathological
cases from tying up the interceptor for minutes. Set to \`null\` to
disable. |
| **\`BufferedOutputStream\` for spill** | Wraps the spill
\`OutputStream\` in 64 KiB \`BufferedOutputStream\`. | Previously every
Tomcat chunk (default 8 KiB) was a separate syscall. Big spilled
responses get a syscall-bound speedup. |
| **Duration timer per phase** | \`payg.filter.duration\` tagged
\`phase=preHandle\` vs \`phase=afterCompletion\`. | Two distinct latency
distributions were blended into one histogram; hard to alert on. |

## Tests

| Test | What it covers |
|---|---|
|
\`PaygOutputExtractorTest.pdfContentType_butBodyMissingPdfMagic_returnsEmpty\`
| New direct-PDF magic-byte gate. |
|
\`PaygChargeInterceptorTest.preHandle_desktopClientHeader_setsJobSourceDesktopApp\`
| New \`X-Stirling-Client: desktop\` → DESKTOP_APP path. |
|
\`PaygChargeInterceptorTest.preHandle_toolId_prefersBestMatchingPattern\`
| Route pattern wins over URI when both are set. |
|
\`PaygChargeInterceptorTest.preHandle_toolId_truncatesAndCountsWhenLongerThan128\`
| Oversized values truncate + increment errors counter. |

Full saas suite green (210 tests), coverage targets met.

## What's NOT in this PR (deliberately)

- **\`?async=true\` OUTPUT recording.** The JobExecutorService returns a
synchronous \`JobResponse{jobId}\` body before the async tool actually
runs; \`afterCompletion\` fires too early. Needs a design decision:
short-circuit PAYG when \`async=true\` OR hook into \`TaskManager\`
completion. Tracked in PR-S4 design doc.
- **JSON-consumes endpoint coverage.** The
\`MultipartHttpServletRequest\` cast skips endpoints with \`consumes =
APPLICATION_JSON_VALUE\` (e.g.
\`ConvertPdfJsonController.exportPartialPdf\`). Fix is either extract a
request-body hash for JSON or add a CI lint forbidding non-multipart
\`@AutoJobPostMapping\`. Design discussion needed.
- **SpringBootTest harness for filter + interceptor wiring.** Saas
module doesn't have one yet. Separate work — PR-S3 takes a different
approach (docker-compose + Behave); a SpringBootTest layer would be
additive in-process coverage.

These are tracked in \`notes/PAYG_DESIGN.md §7.5\` so they don't slip.

## Tracked in

\`notes/PAYG_DESIGN.md\` §7.5 PR-S4.
2026-06-10 09:08:43 +00:00
stirlingbot[bot]andAnthony Stirling be914c7135 Sync Translations + tauri fix for get info (#6484)
### Description of Changes

This Pull Request was automatically generated to synchronize updates to
translation files and documentation. Below are the details of the
changes made:

#### **1. Synchronization of Translation Files**
- Updated translation files
(`frontend/editor/public/locales/*/translation.toml`) to reflect changes
in the reference file `en-GB/translation.toml`.
- Ensured consistency and synchronization across all supported language
files.
- Highlighted any missing or incomplete translations.
- **Format**: TOML

#### **2. Update README.md**
- Generated the translation progress table in `README.md` using
`counter_translation_v3.py`.
- Added a summary of the current translation status for all supported
languages.
- Included up-to-date statistics on translation coverage.

#### **Why these changes are necessary**
- Keeps translation files aligned with the latest reference updates.
- Ensures the documentation reflects the current translation progress.

---

Auto-generated by [create-pull-request][1].

[1]: https://github.com/peter-evans/create-pull-request

---------

Co-authored-by: stirlingbot[bot] <195170888+stirlingbot[bot]@users.noreply.github.com>
Co-authored-by: Anthony Stirling <anthony@stirlingpdf.com>
2026-06-09 21:31:33 +01:00
Anthony Stirling 71361f0d33 Minor: Office doc changes (#6571) 2026-06-09 18:00:34 +01:00
Anthony Stirling 6478c400db Fix font loss in rearrange/overlay/autosplit/OCR from PDFBox (#6545) 2026-06-09 18:00:20 +01:00
Anthony Stirling 502f6c1e4d fix folder causing 500 toast when deleted on another machine (#6551) 2026-06-09 18:00:02 +01:00
Anthony Stirling 1a0beaffc2 stop background flash on tab switches, unblock Audit/Usage demos (#6562) 2026-06-09 17:59:53 +01:00
Anthony Stirling 1e739b6f6f SaaS-aware API landing page (#6585)
# Description of Changes

OLD  (and still current in selfhosted)
<img width="610" height="869" alt="image"
src="https://github.com/user-attachments/assets/f8019298-b4ee-4a68-b928-a9746b64ac1c"
/>


New (in SaaS mode)

<img width="635" height="876" alt="image"
src="https://github.com/user-attachments/assets/6ee4946f-1d7b-42ec-a6f7-75e85739e348"
/>


---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-06-09 14:49:20 +00:00
ConnorYoh 98967bfa86 PAYG: V14 + V15 — subscription_id, free-tier, RPCs, audit logs (#6532)
## Summary

Two Flyway migrations + matching JPA entity updates. **Part 1 of 2** in
the Stripe/Supabase wire-up (PR-SB-1 in
`payg-stripe-supabase-plan.html`); the companion SaaS PR carries the
twin Supabase migrations + new edge functions.

### V14 — payg_subscription_state.sql

- `payg_team_extensions.payg_subscription_id` — the single switch that
decides whether a team is billed. NULL = free-tier or block; NOT NULL =
post Stripe meter events.
- `pricing_policy.free_tier_units_per_cycle` — per-policy free allowance
before a card is required. Default 0.
- `payg_link_subscription(team_id, customer_id, sub_id)` RPC —
idempotent.
- `payg_unlink_subscription(team_id, reason)` RPC — called on
`subscription.deleted`.
- AFTER-INSERT trigger on `teams` so every new signup gets a
`payg_team_extensions` sidecar row from creation.
- Backfill for existing teams without a sidecar row.
- RLS: SELECT permissive (any team member), UPDATE restricted to LEADER.
Service-role bypasses (backend reads + day-1 migration writes).

### V15 — payg_audit_logs.sql

- `payg_meter_event_log` — backend audit of every Stripe meter event
POST attempt (idempotency-key UNIQUE; index on unposted rows for nightly
reconcile).
- `payg_subscription_change_log` — written by V14 RPCs on every
link/unlink.

### Entity updates

- `PaygTeamExtensions.paygSubscriptionId` — read-only field; RPC
functions are the only writers.
- `PricingPolicy.freeTierUnitsPerCycle` — read by upcoming
`PaygTeamUsageService` (PR-SB-4).

### Behaviour change

**None yet.** The columns + functions sit unused until PR-SB-4 wires
`PaygMeterReportingService` and the free-tier gate into
`JobChargeService`. This PR is pure schema + JPA wiring.

## Test plan

- [x] `./gradlew :saas:test` — BUILD SUCCESSFUL
- [x] Manual schema review: column types, FK directions, RLS scope
- [ ] Apply against v3-Supabase via `supabase db push` (after companion
SaaS PR merges)
- [ ] Smoke-test the trigger: `INSERT INTO teams(...)` → assert
`payg_team_extensions` row appears
- [ ] Smoke-test RPCs: SQL-only test of `payg_link_subscription` +
`payg_unlink_subscription` produces expected row + audit entries

## References

- `notes/PAYG_DESIGN.md` (revision note 2026-06-03)
- `payg-stripe-supabase-plan.html` §3.1 (RPC functions), §3.5 (RLS),
§3.10 (audit-log tables)
2026-06-09 14:48:05 +00:00
ConnorYoh ff96a80947 PAYG B-3 / S-3: cucumber suite for shadow-mode flows + CI workflow (#6522)
## What this PR is

End-to-end cucumber coverage for the PAYG shadow charging engine (the
filter + interceptor stack from #6519), wired into CI via a new
`docker-compose-tests-saas.yml` workflow that runs only on PAYG-touching
PRs.

Stacked on #6519.

## Automated scenarios (run by `docker-compose-tests-saas.yml`)

See
[`testing/cucumber/features/payg/shadow_charges.feature`](../tree/payg-s3-cucumber/testing/cucumber/features/payg/shadow_charges.feature):

| Scenario | Validates |
|---|---|
| First tool call writes a CHARGED row | Filter + interceptor fire
end-to-end |
| Lineage join — second call on output | `JobService.joinOrOpen`
matching; no new shadow row |
| 4xx leaves the row CHARGED | "Customer paid for the attempt" semantics
|
| ZIP-returning tool records per-PDF OUTPUT | `PaygOutputExtractor`
unpacks + records signatures |
| Multi-file input writes a single shadow row | Multi-input group sizing
|
| `X-Stirling-Automation` sets PIPELINE source | Header → `JobSource`
detection |

All 6 run locally via `./testing/test-payg.sh` and will run on CI for
any PR that touches `app/saas/**`, the PAYG cucumber features, the saas
compose stack, or the workflow itself.

## Manual-only scenarios — documented in design doc, not in this suite

Two parts of the shadow engine are deliberately not automated; the
engine paths are unit-tested in
`PaygChargeInterceptorTest.afterCompletion_5xx_opened_*`, and the manual
procedures (which require a temporary throw endpoint or a container
restart with a flag flipped) live in [`notes/PAYG_DESIGN.md` §7.5.2
"PAYG cucumber: manual-only
scenarios"](../tree/payg-s3-cucumber/notes/PAYG_DESIGN.md).

- **5xx first-step failure → REFUNDED + CLOSED.** No reliably-5xx-ing
endpoint exists; manual procedure adds a throw endpoint, runs, asserts,
removes.
- **Kill-switch (`PAYG_FILTER_ENABLED=false`).** Needs a container
restart mid-suite; manual procedure tears down, flips env, brings up,
asserts zero shadow rows.

If either gets a hot-reload path (test-only throw endpoint shipped
behind a profile gate, or admin endpoint for the kill switch), automate
it in a follow-up and drop the manual procedure.

## CI workflow

`.github/workflows/docker-compose-tests-saas.yml` (new) —
self-contained, not wired into `build.yml`'s `files-changed` matrix so
the saas-cucumber job fails and succeeds independently. Triggers only on
PAYG-relevant paths. No JaCoCo coverage in v1 (saas compose doesn't have
the coverage override; can add later).

## Test infrastructure (recap)

- **`testing/compose/docker-compose-saas.yml`** — Stirling-PDF backend
with `STIRLING_FLAVOR=saas` + Postgres holding the `stirling_pdf`
schema. Supabase JWT auto-config disabled; API-key auth via
`SECURITY_CUSTOMGLOBALAPIKEY` is the live path the cucumber tests
exercise.
- **`testing/compose/payg/saas-init.sql`** + **`saas-seed.sql`** —
schema bootstrap + idempotent seed (team / user / wallet_policy).
- **`testing/cucumber/features/payg/shadow_charges.feature`** — the 6
scenarios above.
- **`testing/cucumber/features/steps/payg_step_definitions.py`** — step
defs using `requests` (HTTP) + `psycopg` (direct DB inspection). Direct
DB reads are deliberate — we want to see the filter's side effects, not
relay them through another API layer.
- **`testing/test-payg.sh`** — companion runner to `testing/test.sh`.
Brings up the saas compose, waits for health, seeds, runs behave, tears
down.
- **`behave.ini`** excludes `features/payg` from the default behave run
(the saas-cucumber CI job invokes it explicitly).

## Why a separate harness from `testing/test.sh`

The existing `test.sh` covers the proprietary-flavour stack (no PAYG
tables, no saas profile). Coupling two CI matrices that fail and succeed
independently into one script is asking for trouble. Keep the
saas-cucumber job focused on its own concerns; once the harness is
mature, the wider team can decide whether to merge them.

## Tracked in

`notes/PAYG_DESIGN.md` §7.5 (PR-S3) + §7.5.2 (manual scenarios).
2026-06-09 14:47:40 +00:00
Anthony Stirling 347ae9ebbf fix many UI issues (#6569)
# Description of Changes

- Tool action button truncation - fixed by allowing Mantine <Button>
label to wrap (whiteSpace: normal, height: auto) instead of clipping
- Role badge truncation on People page - fixed by dropping the column's
fixed w={100} and letting the badge size to its content
- Settings nav item wraps to 3 lines - fixed by hiding the inline ALPHA
badge by default and revealing it on :hover/:focus-within/.active
- Zoom slider cramped on narrow desktop - fixed by removing the
toolbar's hardcoded minWidth: 30rem and giving the slider flexShrink: 0
+ minWidth: 6rem
- "Swipe left or right" hint on desktop - fixed by adding a useIsTouch()
hook (pointer: coarse) and gating the hint on isMobile && isTouch
- Logout doesn't redirect - fixed by replacing navigate('/login') with
window.location.assign('/login') in a finally block so auth context
fully re-bootstraps
- Viewer top toolbar clips icons on mobile - fixed by switching the
wrapped state to justify-content: flex-start + overflow-x: auto so the
icon strip is momentum-scrollable
- Mobile bottom toolbar overflows - fixed by gating layout on
useIsPhone() and reducing the inline bar to prev / page / next / ⋮ only
- Lost controls when shrinking mobile toolbar - fixed by adding a
Mantine <Menu> behind ⋮ that groups First/Last page, Zoom in/out (with
%), Dual-page, Dark/Sepia filter under Page navigation / Zoom / View
labels
- "Upload from computer" label clipped on hover - fixed by unmounting
the Add Files button entirely while Upload is hovered, so Upload claims
width: 100%
- Settings rows clip controls off-screen - fixed by adding flex: 1,
minWidth: 0 to the inner text-block <div> on 44 rows across 10 -files,
so labels shrink and wrap while controls stay anchored to the right

---
Screenshots 

[report-before-after.html](https://github.com/user-attachments/files/28687621/report-before-after.html)

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-06-09 13:18:57 +00:00
Anthony Stirling 800a411167 Hide endpoints (#6586)
# Description of Changes

Hides the /api/v1/credits endpoints from the generated OpenAPI/Swagger
docs. The root GET /api/v1/credits and GET /api/v1/credits/usage now
carry @Hidden (the 8 admin credit endpoints were already hidden), so the
whole Credit Management controller is gone from the docs.

Adds a single global AI tag to the OpenAPI definition.


Why
We don't want the credits or AI endpoints surfaced in the public API
docs yet as they are not ready for public use, but we do want the AI
endpoints pre-grouped under one AI tag so they land cleanly when we
later un-hide them but dont clutter PDF APIs.

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-06-09 13:18:36 +00:00
Anthony Stirling 66f431a2b7 Lazy-load Stripe SDK so it only loads on checkout (#6546)
# Description of Changes

- Stripe SDK (`@stripe/*` + `js.stripe.com/v3`) was loading on every
page; now it only loads when an upgrade/checkout modal actually opens.
- Converted every import site to `React.lazy()` + `<Suspense>`, gated by
the existing `opened` state.
- Adds a Playwright spec that asserts no Stripe requests on landing or
settings.



---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-06-09 12:36:48 +00:00
James Brunton 0e3cbb3cf2 Explicitly test for console warnings & errors (#6502)
# Description of Changes
Disallow warnings and errors from being thrown in the browser console
during tests unless explicitly expected in the test. Also adds a
Playwright test to prod around some main UI areas and checks that no
warnings/errors have been thrown.
2026-06-09 08:34:02 +00:00
Anthony Stirling 92376b7382 fix: prettier format on AppConfigModal 2026-06-08 21:28:13 +01:00
Anthony Stirling 1d5ce8a1d2 chore: shorten verbose block comments across SaaS branch 2026-06-08 18:38:00 +01:00
Anthony Stirling 8b2baaf0a0 Merge remote-tracking branch 'origin/saas-docker-split' into SaaS 2026-06-08 18:10:02 +01:00
Anthony Stirling d9651f7065 fix(engine): match Dockerfile layout to root Taskfile dir: engine 2026-06-08 18:02:02 +01:00
James Brunton 002de06411 Fix desktop app not being able to load pdfium (#6575)
# Description of Changes
The changes in
[#6279](https://github.com/Stirling-Tools/Stirling-PDF/pull/6279) broke
the desktop app because the wasm URL handling didn't deal with
`tauri://` paths. Also I noticed that `task desktop:build:dev:mac`
failed locally because it was attempting to sign the app with
credentials that developers won't have (and shouldn't need), so I fixed
that too.
2026-06-08 16:21:56 +00:00
Anthony Stirling 4cd03be87a fix: send Supabase token on raw fetch in SaaS chat 2026-06-08 16:41:09 +01:00
Anthony Stirling 02d923f378 chore: remove env var debug from vite.config 2026-06-08 16:29:47 +01:00
Anthony Stirling e7d3430134 merge: pull latest main into SaaS 2026-06-08 16:28:14 +01:00
Anthony Stirling 4b2be58fab debug: fuzzy-match env var names that look like VITE_API_BASE_URL 2026-06-08 16:18:19 +01:00
Anthony Stirling 290c8c2c8b debug: enumerate VITE_/RUN_ env var names in build log 2026-06-08 16:06:56 +01:00
Anthony Stirling 90d6ecd7e1 debug: log env vars at build, write build-info.txt with masked markers 2026-06-08 15:55:40 +01:00
Anthony Stirling a0b7daca52 trigger: rebuild after VITE_API_BASE_URL scope fix 2026-06-08 13:37:30 +01:00
James Brunton 51478e5051 Policies backend (#6527)
# Description of Changes
Add a backend for running any multi-step PDF operations. This is
designed to be used for the upcoming Policies feature, along with
anything else that will require automated running of PDF operations,
like the Automate tool or Processing Folders.

The implementation is not complete. I've tried to get all the
infrastructure in there so that we can add in whichever triggers we need
in the future (like cron triggers or watching folders on disk) but
currently it just supports manual triggering of the policy.

The basis of this work was the operation running from the Stirling
Engine, which this PR removes in favour of this new system. The only
currently accessible frontend way to test this work is to ask the AI
chat to execute multiple operations on a PDF, but I've also extensively
tested with direct API calls to make sure that the policies work and
persist properly.
2026-06-08 10:50:55 +00:00
Anthony Stirling 69e62d8949 exclude unused Redis auto-config (#6547)
fix(health): exclude unused Redis auto-config so default
/actuator/health stays UP

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-06-08 10:41:38 +00:00
Anthony Stirling 2f6b113a13 feat(settings): link ENTERPRISE badges to plan page (#6560)
## Summary
Make the remaining static ENTERPRISE badges in the admin settings
clickable so they navigate the user to `/settings/adminPlan`, matching
the pattern already used by the PRO badges in Connections / Features /
General sections.

### Before
Two ENTERPRISE badges were inert text chips with no affordance:
- `AdminSecuritySection.tsx` - Audit Logging
- `AdminDatabaseSection.tsx` - Database section header

### After
Both now use the same pattern as the existing clickable PRO badges:
- `cursor: pointer`
- `onClick={() => navigate("/settings/adminPlan")}`
- `title` tooltip with the existing
`admin.settings.badge.clickToUpgrade` i18n key ("Click to view plan
details")

No new strings, no new components - just wiring up existing behavior to
the two badges that were missing it.

### Existing already-clickable badges (kept identical for reference)
- `AdminConnectionsSection.tsx:585-596` - SSO Auto Login PRO
- `AdminFeaturesSection.tsx:175-186` - Server Certificate PRO
- `AdminGeneralSection.tsx:920-931` - Custom Metadata PRO
2026-06-08 10:40:35 +00:00
Anthony Stirling af52134811 fix(automate): flip AutomationEntry tooltip to position=left (#6550)
# Description of Changes

automate description tooltip was facing wrong way

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-06-08 10:38:00 +00:00
Anthony Stirling 8a2474ff60 fix(desktop): enable in-page drag-drop in Tauri build (#6548)
## Summary

- Set `dragDropEnabled: false` on the Tauri window so HTML5 drag events
reach the WebView. Previously the default `true` made Tauri intercept
all drag-drop at the OS level, silently breaking in-page drag-to-reorder
(Pragmatic Drag and Drop in `FileEditorThumbnail` /
`useFileItemDragDrop`) in the desktop build. The Active Files tab
reorder, which feeds Merge ordering, was the user-visible symptom.
- Browser builds are unaffected (tauri.conf.json is desktop-only).
- The OS file-drop pipeline now flows through the existing Mantine
`Dropzone` in `FileEditor.tsx` via HTML5 events instead of the Rust
`WindowEvent::DragDrop` handler in `lib.rs:215`. Verified working.

## Test plan

- [x] Desktop: drag a thumbnail in Active Files past another - row goes
semi-transparent, order updates on drop.
- [x] Desktop: drag a PDF from File Explorer onto the window - file is
added.
- [x] Web build: drag-to-reorder still works (unchanged code path; flag
is desktop-only).
- [x] Merge tool: order set by drag in Active Files is the order used by
the merge output.

## Follow-up (not in this PR)

- `WindowEvent::DragDrop` arm in
`frontend/editor/src-tauri/src/lib.rs:215-229` is now unreachable for
window drops. The `forward_files_to_window` helper still serves the
macOS Finder "Open With" path (`RunEvent::Opened` at lib.rs:230), so
only the DragDrop arm can be deleted. Worth a small cleanup pass later.
2026-06-08 10:36:58 +00:00
Anthony Stirling d202c9c32f Minor: sanitize SVG (#6572)
# Description of Changes

Sanitize SVG

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-06-08 10:31:41 +00:00
Anthony Stirling 1ef03c43b4 fix(i18n): wrap hard-coded English strings in t() across UI (#6566)
## Summary
Audit + bulk fix of hard-coded English UI strings - `aria-label`,
`title`, `placeholder`, `label`, and raw JSX literals that bypassed i18n
entirely. Each literal now goes through `t("key", "English Default")`
from `react-i18next`, and every new key has a corresponding entry in
`en-GB/translation.toml` so translators can pick it up.

## What this fixes
Strings were rendered untranslated in every non-EN locale because they
never went through `t()` at all (not just "value not translated yet").
Affects screen-reader labels, tooltips, form placeholders, empty/loading
states, plan card content, and the entire workflow ParticipantView.

## Coverage (~143 keys / 50 files)
- **Viewer chrome** - search bar (close, clear, prev/next, "of N"
results), link/signature/redaction actions, viewer error state, zoom
labels
- **Page editor** - undo/redo/rotate/delete toolbar tooltips, empty
state, bulk selection operator chip tooltips
- **Shared primitives** - Tooltip close, InfoBanner dismiss, TextInput
clear, Toast dismiss/toggle, UpdateModal close, EditableSecretField
edit, DropdownListWithFooter search, FileCard/FileDropdownMenu actions,
EmptyFilesState + AddFileCard upload
- **Tools** - Image upload + hint, ColorControl eyedropper, sign Use
Signature, CompressSettings, OCR loading, PageLayout
margin/border/row/col placeholders, FormFill switch + save + re-scan
- **Proprietary admin** - OverviewHeader signed-in line + logout,
AdminPremiumSection moved-features list (via `<Trans>`),
AdminPlanSection no-data alert, AdminAdvancedSection temp-dir
placeholders, AdminEndpointsSection multiselect placeholders,
AdminMailSection + AdminDatabaseSection password placeholders
- **Onboarding** - MFASetupSlide QR loading + auth code label,
SecurityCheckSlide role select + options
- **ParticipantView** - entire sign-document UI (~30 strings: loading,
error, badges, headings, cert-type Select, all input labels and
placeholders, action buttons, completion + expired alerts) - file
previously imported `useTranslation` but only used `t()` for cert
validation
- **planConstants.ts refactor** - replaced `PLAN_FEATURES` /
`PLAN_HIGHLIGHTS` const exports with `usePlanFeatures()` /
`usePlanHighlights()` hooks. Service layer (`licenseService.getPlans`)
updated to accept feature/highlight maps so it stays hook-free. Callers
(`usePlans`, `CheckoutContext`) resolve the hooks at the React boundary
- **Previously catalogued offenders** - `FileSidebarFileItem`
open/close-viewer aria-labels, `quickAccessBar/ActiveToolButton` "Back
to all tools" tooltip + aria, `AppConfigModal` close button

## Notes
- One small refactor in `usePageSelectionTips.ts` was needed to resolve
a TOML key-shape conflict: the existing scalar keys
`bulkSelection.operators.{and,not,comma}` needed to become tables to
hold the new `.title` subkeys for OperatorsSection's chip tooltips. The
existing descriptions moved to `[bulkSelection.operators.descriptions]`
and the three i18n key paths in usePageSelectionTips were updated to
match.
- Viewer sidebar close buttons
(Bookmark/Layer/Thumbnail/Attachment/Comments) were on the audit list
but are NOT on main - they're added by the unmerged PR #6552
(feat/viewer-sidebar-ux). Those particular strings will need wrapping
when that PR lands.
- TOML hook (`toml-sort-fix`) ran and re-sorted the translation file.

## Test plan
- [ ] `task frontend:typecheck` passes (core + proprietary + desktop
variants)
- [ ] `task frontend:lint` passes
- [ ] Switching language to Deutsch / Русский: previously-English
`aria-label`s + tooltips + placeholders + plan card bullets now render
translated (when the locale has values) or fall back to the English
default (when it doesn't)
- [ ] Plan page bullet points in EN render unchanged
- [ ] Sign-document flow (ParticipantView) renders unchanged in EN
2026-06-08 10:11:43 +00:00
Anthony Stirling 0b575ed841 fix: respect BASE_PATH in AI chat fetch and pdfjs worker assets 2026-06-06 21:21:24 +01:00
Anthony Stirling 940cb2fc44 chore: trigger Cloudflare deploy 2026-06-06 19:26:08 +01:00
Anthony Stirling 9da0a0d020 fix: respect BASE_PATH in redirects, comparisons, and cookie consent paths 2026-06-06 19:20:07 +01:00
Anthony Stirling 0b944a29a7 Prefer Maven Central over jboss/shibboleth mirrors for resilience 2026-06-03 09:10:08 +01:00
Anthony Stirling 58aeba2bf7 Add backend-only and SaaS-aware frontend Dockerfiles 2026-06-03 09:03:58 +01:00
1055 changed files with 126455 additions and 9603 deletions
+13 -1
View File
@@ -1,6 +1,8 @@
build: &build
- build.gradle
- app/(common|core|proprietary)/build.gradle
- Taskfile.yml
- .taskfiles/backend.yml
openapi: &openapi
- *build
@@ -38,6 +40,9 @@ project: &project
- frontend/**
- docker/**
- scripts/RestartHelper.java
- Taskfile.yml
- .taskfiles/backend.yml
- .taskfiles/docker.yml
- scripts/db-migration/**
- .github/workflows/db-migration-test.yml
@@ -55,6 +60,9 @@ frontend: &frontend
- scripts/summarize_type3_signatures.py
- scripts/type3_to_cff.py
- scripts/update_type3_library.py
- Taskfile.yml
- .taskfiles/frontend.yml
- .taskfiles/e2e.yml
# Files that affect the Tauri desktop bundle. Gate the multi-OS Tauri build
# job on changes to any of these.
@@ -66,6 +74,8 @@ tauri: &tauri
- frontend/package-lock.json
- frontend/editor/vite.config.ts
- .github/workflows/tauri-build.yml
- Taskfile.yml
- .taskfiles/desktop.yml
# Files that affect the AI engine (Python tool models, fixers, tests). Gate
# the engine validation job on changes to engine sources or to the Java
@@ -74,6 +84,8 @@ engine: &engine
- engine/**
- app/(common|core|proprietary)/src/main/java/**
- .github/workflows/ai-engine.yml
- Taskfile.yml
- .taskfiles/engine.yml
licenses-frontend: &licenses-frontend
- ".github/workflows/frontend-backend-licenses-update.yml"
@@ -102,4 +114,4 @@ proprietary: &proprietary
- configs/settings.yml.template
- build.gradle
- app/proprietary/build.gradle
- .github/workflows/build-enterprise.yml
- .github/workflows/build-enterprise.yml
+3 -3
View File
@@ -13,7 +13,7 @@ Usage:
"""
# Sample for Windows:
# python .github/scripts/check_language_toml.py --reference-file frontend/editor/public/locales/en-GB/translation.toml --branch "" --files frontend/editor/public/locales/de-DE/translation.toml frontend/editor/public/locales/fr-FR/translation.toml
# python .github/scripts/check_language_toml.py --reference-file frontend/editor/public/locales/en-US/translation.toml --branch "" --files frontend/editor/public/locales/de-DE/translation.toml frontend/editor/public/locales/fr-FR/translation.toml
import argparse
import glob
@@ -211,7 +211,7 @@ def check_for_differences(reference_file, file_list, branch, actor):
)
continue
if basename_current_file == basename_reference_file and locale_dir == "en-GB":
if basename_current_file == basename_reference_file and locale_dir == "en-US":
continue
if (
@@ -308,7 +308,7 @@ def check_for_differences(reference_file, file_list, branch, actor):
report.append("## ❌ Overall Check Status: **_Failed_**")
report.append("")
report.append(
f"@{actor} please check your translation if it conforms to the standard. Follow the format of [en-GB/translation.toml](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/frontend/editor/public/locales/en-GB/translation.toml)"
f"@{actor} please check your translation if it conforms to the standard. Follow the format of [en-US/translation.toml](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/frontend/editor/public/locales/en-US/translation.toml)"
)
else:
report.append("## ✅ Overall Check Status: **_Success_**")
+2 -2
View File
@@ -239,7 +239,7 @@ jobs:
- name: Build and push V2 image (Depot)
if: env.USE_DEPOT == 'true' && steps.check-image.outputs.exists == 'false'
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.16.0
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: .
@@ -293,7 +293,7 @@ jobs:
SECURITY_ENABLELOGIN: "true"
SECURITY_INITIALLOGIN_USERNAME: "${{ secrets.TEST_LOGIN_USERNAME }}"
SECURITY_INITIALLOGIN_PASSWORD: "${{ secrets.TEST_LOGIN_PASSWORD }}"
SYSTEM_DEFAULTLOCALE: en-GB
SYSTEM_DEFAULTLOCALE: en-US
UI_APPNAME: "Stirling-PDF V2 PR#${{ needs.check-pr.outputs.pr_number }}"
UI_HOMEDESCRIPTION: "V2 PR#${{ needs.check-pr.outputs.pr_number }} - Embedded Architecture"
UI_APPNAMENAVBAR: "V2 PR#${{ needs.check-pr.outputs.pr_number }}"
@@ -222,10 +222,10 @@ jobs:
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
with:
gradle-version: 9.3.1
gradle-version: 9.5.1
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
- name: Run Gradle Command
run: |
if [ "${{ needs.check-comment.outputs.disable_security }}" == "true" ]; then
@@ -256,7 +256,7 @@ jobs:
- name: Build and push PR-specific image (Depot)
if: env.USE_DEPOT == 'true'
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.16.0
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: .
@@ -285,7 +285,7 @@ jobs:
- name: Build and push engine image (Depot)
if: env.USE_DEPOT == 'true' && needs.check-comment.outputs.enable_prototypes == 'true'
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.16.0
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: ./engine
@@ -388,7 +388,7 @@ jobs:
environment:
DISABLE_ADDITIONAL_FEATURES: "${DISABLE_ADDITIONAL_FEATURES}"
SECURITY_ENABLELOGIN: "${LOGIN_SECURITY}"
SYSTEM_DEFAULTLOCALE: en-GB
SYSTEM_DEFAULTLOCALE: en-US
UI_APPNAME: "Stirling-PDF PR#${PR_NUMBER}"
UI_HOMEDESCRIPTION: "PR#${PR_NUMBER} for Stirling-PDF Latest"
UI_APPNAMENAVBAR: "PR#${PR_NUMBER}"
+2 -2
View File
@@ -43,10 +43,10 @@ jobs:
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
with:
gradle-version: 9.3.1
gradle-version: 9.5.1
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
- name: Regenerate tool models
run: task engine:tool-models
+2 -2
View File
@@ -58,11 +58,11 @@ jobs:
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
with:
gradle-version: 9.3.1
gradle-version: 9.5.1
cache-disabled: true
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
- name: Check Java formatting (Spotless)
# Runs once per matrix combination - pick the cheapest leg
# (core - no proprietary, no saas) so we don't wait for the
+1 -1
View File
@@ -78,7 +78,7 @@ jobs:
cache: "npm"
cache-dependency-path: frontend/package-lock.json
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
- name: Install Playwright (chromium only)
run: task e2e:install -- chromium
- name: Build frontend (needed for playwright's vite preview webServer)
+2 -2
View File
@@ -40,11 +40,11 @@ jobs:
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
with:
gradle-version: 9.3.1
gradle-version: 9.5.1
cache-disabled: true
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
- name: Check licenses for compatibility
run: task backend:licenses:check
env:
+2 -2
View File
@@ -45,11 +45,11 @@ jobs:
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
with:
gradle-version: 9.3.1
gradle-version: 9.5.1
cache-disabled: true
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
- name: Generate OpenAPI documentation
run: task backend:swagger
env:
+6 -6
View File
@@ -166,16 +166,16 @@ jobs:
// Determine reference file
let referenceFilePath;
if (changedFiles.includes("frontend/editor/public/locales/en-GB/translation.toml")) {
if (changedFiles.includes("frontend/editor/public/locales/en-US/translation.toml")) {
console.log("Using PR branch reference file.");
const { data: fileContent } = await github.rest.repos.getContent({
owner: prRepoOwner,
repo: prRepoName,
path: "frontend/editor/public/locales/en-GB/translation.toml",
path: "frontend/editor/public/locales/en-US/translation.toml",
ref: branch,
});
referenceFilePath = "pr-branch-translation-en-GB.toml";
referenceFilePath = "pr-branch-translation-en-US.toml";
const content = Buffer.from(fileContent.content, "base64").toString("utf-8");
fs.writeFileSync(referenceFilePath, content);
} else {
@@ -183,11 +183,11 @@ jobs:
const { data: fileContent } = await github.rest.repos.getContent({
owner: repoOwner,
repo: repoName,
path: "frontend/editor/public/locales/en-GB/translation.toml",
path: "frontend/editor/public/locales/en-US/translation.toml",
ref: "main",
});
referenceFilePath = "main-branch-translation-en-GB.toml";
referenceFilePath = "main-branch-translation-en-US.toml";
const content = Buffer.from(fileContent.content, "base64").toString("utf-8");
fs.writeFileSync(referenceFilePath, content);
}
@@ -293,6 +293,6 @@ jobs:
run: |
echo "Cleaning up temporary files..."
rm -rf pr-branch
rm -f pr-branch-translation-en-GB.toml main-branch-translation-en-GB.toml changed_files.txt result.txt
rm -f pr-branch-translation-en-US.toml main-branch-translation-en-US.toml changed_files.txt result.txt
echo "Cleanup complete."
continue-on-error: true # Ensure cleanup runs even if previous steps fail
+1 -1
View File
@@ -48,7 +48,7 @@ jobs:
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
with:
gradle-version: 9.3.1
gradle-version: 9.5.1
cache-disabled: true
# No `-PnoSpotless` here yet because the upstream cache layer matches the
+3 -3
View File
@@ -107,7 +107,7 @@ jobs:
- name: Build and push frontend image (Depot)
if: env.USE_DEPOT == 'true' && steps.check-frontend.outputs.exists == 'false'
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.16.0
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: .
@@ -136,7 +136,7 @@ jobs:
- name: Build and push backend image (Depot)
if: env.USE_DEPOT == 'true' && steps.check-backend.outputs.exists == 'false'
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.16.0
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: .
@@ -188,7 +188,7 @@ jobs:
environment:
DISABLE_ADDITIONAL_FEATURES: "true"
SECURITY_ENABLELOGIN: "false"
SYSTEM_DEFAULTLOCALE: en-GB
SYSTEM_DEFAULTLOCALE: en-US
UI_APPNAME: "Stirling-PDF V2"
UI_HOMEDESCRIPTION: "V2 Frontend/Backend Split"
UI_APPNAMENAVBAR: "V2 Deployment"
+1 -1
View File
@@ -61,7 +61,7 @@ jobs:
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
with:
gradle-version: 9.3.1
gradle-version: 9.5.1
cache-disabled: true
- name: Set up Docker Buildx
+23 -3
View File
@@ -42,7 +42,7 @@ jobs:
cache: "npm"
cache-dependency-path: frontend/package-lock.json
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
- name: Install Playwright (chromium only)
run: task e2e:install -- chromium
- name: Build frontend (production bundle for vite preview)
@@ -188,10 +188,30 @@ jobs:
name: backend-log-live-${{ github.run_id }}
path: .test-state/playwright/backend.log
retention-days: 7
- name: Upload Playwright report
- name: List Playwright output locations (debug)
if: always()
run: |
echo "::group::Playwright output dirs"
# Playwright anchors its default outputDir + HTML report to the
# nearest package.json, which is frontend/ (frontend/editor has
# none), so artifacts land under frontend/, not frontend/editor/.
ls -la frontend/playwright-report 2>/dev/null \
|| echo "no playwright-report at frontend/"
ls -la frontend/test-results 2>/dev/null \
|| echo "no test-results at frontend/"
find . -name node_modules -prune -o -name 'trace.zip' -print 2>/dev/null || true
echo "::endgroup::"
- name: Upload Playwright report + traces
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: playwright-report-live-${{ github.run_id }}
path: frontend/editor/playwright-report/
# test-results/ holds the per-test trace.zip (with browser console
# logs) + screenshots/video; playwright-report/ is the HTML report.
# Both live under frontend/ (Playwright anchors them to the nearest
# package.json, which is frontend/; frontend/editor has none).
path: |
frontend/playwright-report/
frontend/test-results/
retention-days: 7
if-no-files-found: warn
+1 -1
View File
@@ -36,7 +36,7 @@ jobs:
cache: "npm"
cache-dependency-path: frontend/package-lock.json
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
- name: Install Playwright (chromium only)
run: task e2e:install -- chromium
- name: Build frontend (production bundle for vite preview)
@@ -97,7 +97,7 @@ jobs:
run: npm ci --ignore-scripts --audit=false --fund=false
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
- name: Generate frontend license report (internal PR)
if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork == false
env:
@@ -349,10 +349,10 @@ jobs:
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
with:
gradle-version: 9.3.1
gradle-version: 9.5.1
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
- name: Check licenses and generate report
id: license-check
run: task backend:licenses:generate || echo "LICENSE_CHECK_FAILED=true" >> $GITHUB_ENV
+1 -1
View File
@@ -31,7 +31,7 @@ jobs:
cache: "npm"
cache-dependency-path: frontend/package-lock.json
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
- name: Quality-check frontend
id: frontend-check
run: task frontend:check:all
+6 -6
View File
@@ -73,10 +73,10 @@ jobs:
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
with:
gradle-version: 9.3.1
gradle-version: 9.5.1
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
- name: Get version number
id: versionNumber
run: |
@@ -148,7 +148,7 @@ jobs:
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
with:
gradle-version: 9.3.1
gradle-version: 9.5.1
- name: Setup Node.js
if: matrix.variant.build_frontend == true
@@ -159,7 +159,7 @@ jobs:
cache-dependency-path: frontend/package-lock.json
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
- name: Build JAR
run: ./gradlew build ${{ matrix.variant.build_frontend && '-PbuildWithFrontend=true' || '' }} -x spotlessApply -x spotlessCheck -x test -x sonarqube
@@ -252,10 +252,10 @@ jobs:
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
with:
gradle-version: 9.3.1
gradle-version: 9.5.1
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
# Build the universal JRE before desktop:prepare so the jlink:runtime
# task short-circuits on its `test -d runtime/jre` status check.
+1 -1
View File
@@ -37,7 +37,7 @@ jobs:
cache-dependency-path: frontend/package-lock.json
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
- name: Install all Playwright browsers
run: task e2e:install
+1 -1
View File
@@ -75,7 +75,7 @@ jobs:
- name: Generate tags for base image
id: meta
uses: docker/metadata-action@030e881283bb7a6894de51c315a6bfe6a94e05cf # v6.0.0
uses: docker/metadata-action@80c7e94dd9b9319bd5eb7a0e0fe9291e23a2a2e9 # v6.1.0
with:
images: |
${{ secrets.DOCKER_HUB_ORG_USERNAME }}/stirling-pdf-base
+5 -5
View File
@@ -78,14 +78,14 @@ jobs:
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
with:
gradle-version: 9.3.1
gradle-version: 9.5.1
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
- name: Get version number
id: versionNumber
run: echo "versionNumber=$(./gradlew printVersion --quiet | tail -1)" >> $GITHUB_OUTPUT
@@ -129,7 +129,7 @@ jobs:
- name: Generate tags for latest
id: meta
if: env.RUN_MAIN_APP == 'true'
uses: docker/metadata-action@030e881283bb7a6894de51c315a6bfe6a94e05cf # v6.0.0
uses: docker/metadata-action@80c7e94dd9b9319bd5eb7a0e0fe9291e23a2a2e9 # v6.1.0
with:
images: |
${{ secrets.DOCKER_HUB_USERNAME }}/s-pdf
@@ -178,7 +178,7 @@ jobs:
- name: Generate tags for latest-fat
id: meta-fat
uses: docker/metadata-action@030e881283bb7a6894de51c315a6bfe6a94e05cf # v6.0.0
uses: docker/metadata-action@80c7e94dd9b9319bd5eb7a0e0fe9291e23a2a2e9 # v6.1.0
if: env.RUN_MAIN_APP == 'true' && github.ref != 'refs/heads/main' && github.ref != 'refs/heads/testMain'
with:
images: |
@@ -222,7 +222,7 @@ jobs:
- name: Generate tags for ultra-lite
id: meta-lite
uses: docker/metadata-action@030e881283bb7a6894de51c315a6bfe6a94e05cf # v6.0.0
uses: docker/metadata-action@80c7e94dd9b9319bd5eb7a0e0fe9291e23a2a2e9 # v6.1.0
if: env.RUN_MAIN_APP == 'true' && github.ref != 'refs/heads/main' && github.ref != 'refs/heads/testMain'
with:
images: |
+1 -1
View File
@@ -22,7 +22,7 @@ jobs:
egress-policy: audit
- name: 30 days stale issues
uses: actions/stale@b5d41d4e1d5dceea10e7104786b73624c18a190f # v10.2.0
uses: actions/stale@eb5cf3af3ac0a1aa4c9c45633dd1ae542a27a899 # v10.3.0
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
days-before-stale: 30
+2 -2
View File
@@ -48,7 +48,7 @@ jobs:
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
with:
gradle-version: 9.3.1
gradle-version: 9.5.1
- name: Generate Swagger documentation
run: ./gradlew :stirling-pdf:generateOpenApiDocs
@@ -63,7 +63,7 @@ jobs:
SWAGGERHUB_USER: "Frooodle"
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
- name: Get version number
id: versionNumber
run: echo "versionNumber=$(./gradlew printVersion --quiet | tail -1)" >> $GITHUB_OUTPUT
+2 -2
View File
@@ -62,7 +62,7 @@ jobs:
- name: Sync translation TOML files
run: |
python .github/scripts/check_language_toml.py --reference-file "frontend/editor/public/locales/en-GB/translation.toml" --branch main
python .github/scripts/check_language_toml.py --reference-file "frontend/editor/public/locales/en-US/translation.toml" --branch main
- name: pre-commit run
run: |
@@ -100,7 +100,7 @@ jobs:
This Pull Request was automatically generated to synchronize updates to translation files and documentation. Below are the details of the changes made:
#### **1. Synchronization of Translation Files**
- Updated translation files (`frontend/editor/public/locales/*/translation.toml`) to reflect changes in the reference file `en-GB/translation.toml`.
- Updated translation files (`frontend/editor/public/locales/*/translation.toml`) to reflect changes in the reference file `en-US/translation.toml`.
- Ensured consistency and synchronization across all supported language files.
- Highlighted any missing or incomplete translations.
- **Format**: TOML
+2 -2
View File
@@ -136,10 +136,10 @@ jobs:
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
with:
gradle-version: 9.3.1
gradle-version: 9.5.1
- name: Setup Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
- name: Build universal macOS JRE
if: matrix.platform == 'macos-15'
+4 -4
View File
@@ -106,11 +106,11 @@ jobs:
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
with:
gradle-version: 9.3.1
gradle-version: 9.5.1
cache-disabled: true
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
- name: Build application
run: task backend:build
env:
@@ -157,7 +157,7 @@ jobs:
- name: Build ${{ matrix.docker-rev }} (Depot)
if: env.USE_DEPOT == 'true'
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.16.0
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: .
@@ -230,7 +230,7 @@ jobs:
- name: Build docker/unoserver/Dockerfile (Depot)
if: env.USE_DEPOT == 'true'
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.16.0
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: .
+3 -3
View File
@@ -51,7 +51,7 @@ jobs:
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
with:
gradle-version: 9.3.1
gradle-version: 9.5.1
- name: Build with Gradle
run: ./gradlew build
@@ -83,7 +83,7 @@ jobs:
- name: Build and push test image (Depot)
if: env.USE_DEPOT == 'true'
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.16.0
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: .
@@ -129,7 +129,7 @@ jobs:
environment:
DISABLE_ADDITIONAL_FEATURES: "true"
SECURITY_ENABLELOGIN: "false"
SYSTEM_DEFAULTLOCALE: en-GB
SYSTEM_DEFAULTLOCALE: en-US
UI_APPNAME: "Stirling-PDF Test"
UI_HOMEDESCRIPTION: "Test Deployment"
UI_APPNAMENAVBAR: "Test"
+9
View File
@@ -3,3 +3,12 @@
# intentionally in #6150 so engine/.env has a working default, with real
# credentials overridden via engine/.env.local.
engine/.env:generic-api-key:41
# MCP test fixtures / harness - no real secrets:
# - test-only API key constant in an integration test
# - JDBC URL + throwaway Keycloak creds in the local test compose
# - placeholder / shell-variable Bearer headers in curl-based validation scripts
app/proprietary/src/test/java/stirling/software/proprietary/mcp/security/McpApiKeyIntegrationTest.java:generic-api-key:40
testing/compose/docker-compose-keycloak-mcp.yml:generic-api-key:25
testing/compose/validate-mcp-apikey.sh:curl-auth-header:73
testing/compose/validate-mcp-test.sh:curl-auth-header:92
+14
View File
@@ -18,6 +18,15 @@ version: '3'
tasks:
dev:
desc: "Start backend dev server"
cmds:
- task: dev:proprietary
vars:
PORT: '{{.PORT}}'
AIENGINE_URL: '{{.AIENGINE_URL}}'
AIENGINE_TIMEOUTSECONDS: '{{.AIENGINE_TIMEOUTSECONDS}}'
dev:proprietary:
desc: "Start backend dev server in proprietary mode"
ignore_error: true
vars:
PORT: '{{.PORT | default "8080"}}'
@@ -50,9 +59,14 @@ tasks:
PORT: '{{.PORT | default "8080"}}'
# Override to "" to run the pure `saas` profile against your own SAAS_DB_*.
PROFILES: '{{.PROFILES | default "dev"}}'
AIENGINE_URL: '{{.AIENGINE_URL | default ""}}'
AIENGINE_TIMEOUTSECONDS: '{{.AIENGINE_TIMEOUTSECONDS | default "120"}}'
env:
SERVER_PORT: '{{.PORT}}'
STIRLING_FLAVOR: saas
AIENGINE_URL: '{{.AIENGINE_URL}}'
AIENGINE_ENABLED: '{{if .AIENGINE_URL}}true{{else}}false{{end}}'
AIENGINE_TIMEOUTSECONDS: '{{.AIENGINE_TIMEOUTSECONDS}}'
cmds:
- cmd: cmd /c ".\gradlew.bat :stirling-pdf:bootRun {{if .PROFILES}}--args=\"--spring.profiles.include={{.PROFILES}}\"{{end}}"
platforms: [windows]
+15 -12
View File
@@ -1,7 +1,9 @@
version: '3'
vars:
JLINK_MODULES: "java.base,java.compiler,java.desktop,java.instrument,java.logging,java.management,java.naming,java.net.http,java.prefs,java.rmi,java.scripting,java.security.jgss,java.security.sasl,java.sql,java.transaction.xa,java.xml,java.xml.crypto,jdk.crypto.ec,jdk.crypto.cryptoki,jdk.unsupported"
# jdk.dynalink is required by VeraPDF (PDF/A validation); without it the bundled JRE throws
# NoClassDefFoundError: jdk/dynalink/Namespace at runtime in get-info-on-pdf and verify-pdf
JLINK_MODULES: "java.base,java.compiler,java.desktop,java.instrument,java.logging,java.management,java.naming,java.net.http,java.prefs,java.rmi,java.scripting,java.security.jgss,java.security.sasl,java.sql,java.transaction.xa,java.xml,java.xml.crypto,jdk.crypto.ec,jdk.crypto.cryptoki,jdk.unsupported,jdk.dynalink"
# Override via JPDFIUM_PLATFORMS env (csv of platform keys, or 'all').
JPDFIUM_PLATFORMS:
@@ -62,21 +64,21 @@ tasks:
deps: [prepare]
dir: editor
cmds:
- npx tauri build --bundles app
- npx tauri build --bundles app --config '{"bundle":{"createUpdaterArtifacts":false}}'
build:dev:windows:
desc: "Build Tauri desktop NSIS installer (Windows)"
deps: [prepare]
dir: editor
cmds:
- npx tauri build --bundles nsis
- npx tauri build --bundles nsis --config '{"bundle":{"createUpdaterArtifacts":false}}'
build:dev:linux:
desc: "Build Tauri desktop AppImage (Linux)"
deps: [prepare]
dir: editor
cmds:
- npx tauri build --bundles appimage
- npx tauri build --bundles appimage --config '{"bundle":{"createUpdaterArtifacts":false}}'
test:
desc: "Run Tauri/Cargo tests"
@@ -125,14 +127,15 @@ tasks:
cmds:
- rm -rf runtime/jre
- mkdir -p runtime
- >-
jlink
--add-modules {{.JLINK_MODULES}}
--strip-debug
--compress=zip-6
--no-header-files
--no-man-pages
--output runtime/jre
- |
JLINK_COMPRESS="$(jlink --help 2>&1 | grep -q 'zip-\[0-9\]' && echo zip-6 || echo 2)"
jlink \
--add-modules {{.JLINK_MODULES}} \
--strip-debug \
--compress="$JLINK_COMPRESS" \
--no-header-files \
--no-man-pages \
--output runtime/jre
# jlink emits its files mode 444 (read-only). Tauri's build-script
# resource copier preserves source permissions when staging
# `runtime/jre/**/*` into `target/<profile>/runtime/jre/...`, so the
+5
View File
@@ -20,6 +20,11 @@ tasks:
cmds:
- docker build -t stirling-pdf-ultra-lite -f {{.EMBEDDED_DIR}}/Dockerfile.ultra-lite .
build:backend:
desc: "Build backend-only Docker image (no embedded frontend)"
cmds:
- docker build -t stirling-pdf-backend -f docker/backend/Dockerfile .
build:frontend:
desc: "Build frontend-only Docker image"
cmds:
+52
View File
@@ -212,3 +212,55 @@ tasks:
desc: "Stop the SAML keycloak test environment"
cmds:
- docker compose -f testing/compose/docker-compose-keycloak-saml.yml down -v
mcp:up:
desc: "Start the MCP keycloak test environment (Stirling as OAuth resource server)"
summary: |
Brings up Keycloak (OAuth authorization server) + Stirling configured as an
MCP resource server, then you can exercise /mcp with real Keycloak tokens.
Set LICENSE_KEY=<KEY> to skip the interactive license prompt:
task e2e:mcp:up LICENSE_KEY=abc123
Pass extra flags via -- :
task e2e:mcp:up -- --validate --nobuild
ignore_error: true
cmds:
- bash testing/compose/start-mcp-test.sh {{if .LICENSE_KEY}}--license-key "{{.LICENSE_KEY}}"{{end}} {{.CLI_ARGS}}
mcp:manual:
desc: "Start the MCP keycloak test env in manual mode (prints URLs + a live token for your client)"
summary: |
Brings the stack up and prints copy-paste URLs/commands plus a freshly minted
access token so you can drive your own MCP client (Inspector, curl, ...).
task e2e:mcp:manual LICENSE_KEY=<your-license-key>
Add --nobuild if the images are already built:
task e2e:mcp:manual LICENSE_KEY=<your-license-key> -- --nobuild
ignore_error: true
cmds:
- bash testing/compose/start-mcp-test.sh --manual {{if .LICENSE_KEY}}--license-key "{{.LICENSE_KEY}}"{{end}} {{.CLI_ARGS}}
mcp:apikey:
desc: "Start the MCP test env in API-KEY manual mode (no OAuth/IdP): mints a key + prints client settings"
summary: |
Brings Stirling up in apikey auth mode and prints copy-paste client settings with a freshly
minted X-API-KEY - ideal for clients whose OAuth layer can't reach localhost.
task e2e:mcp:apikey LICENSE_KEY=<your-license-key>
Add --nobuild if images are already built:
task e2e:mcp:apikey LICENSE_KEY=<your-license-key> -- --nobuild
ignore_error: true
cmds:
- bash testing/compose/start-mcp-test.sh --apikey {{if .LICENSE_KEY}}--license-key "{{.LICENSE_KEY}}"{{end}} {{.CLI_ARGS}}
mcp:validate:
desc: "Validate the running MCP keycloak test environment end-to-end (oauth mode + real MCP SDK client)"
cmds:
- bash testing/compose/validate-mcp-test.sh
mcp:validate-apikey:
desc: "Validate the MCP server in API-KEY auth mode (mints a key + real MCP SDK client), then restore oauth"
cmds:
- bash testing/compose/validate-mcp-apikey.sh
mcp:down:
desc: "Stop the MCP keycloak test environment"
cmds:
- docker compose -f testing/compose/docker-compose-keycloak-mcp.yml down -v
+1 -1
View File
@@ -33,7 +33,7 @@ tasks:
env:
PYTHONUNBUFFERED: "1"
cmds:
- uv run uvicorn stirling.api.app:app --host 0.0.0.0 --port {{.PORT}}
- uv run uvicorn stirling.api.app:app --host 0.0.0.0 --port {{.PORT}} --workers "${STIRLING_ENGINE_WORKERS:-4}"
dev:
desc: "Start engine dev server with hot reload"
+3 -3
View File
@@ -327,19 +327,19 @@ tasks:
test:
desc: "Run tests"
deps: [install]
deps: [prepare]
cmds:
- npx vitest run --root editor
test:watch:
desc: "Run tests in watch mode"
deps: [install]
deps: [prepare]
cmds:
- npx vitest --watch --root editor
test:coverage:
desc: "Run tests with coverage (one-shot; CI-friendly)."
deps: [install]
deps: [prepare]
cmds:
# `vitest run` makes this CI-safe (the bare `vitest` form enters watch
# mode). Explicit reporter list because v8 + json-summary is what the
+3 -3
View File
@@ -200,9 +200,9 @@ const [ToolName] = (props: BaseToolProps) => {
```
## 5. Add Translations
Update translation files. **Important: Only update `en-GB` files** - other languages are handled separately.
Update translation files. **Important: Only update `en-US` files** - other languages are handled separately.
**File to update:** `frontend/editor/public/locales/en-GB/translation.toml`
**File to update:** `frontend/editor/public/locales/en-US/translation.toml`
**Required Translation Keys**:
```toml
@@ -251,7 +251,7 @@ Update translation files. **Important: Only update `en-GB` files** - other langu
```
**Translation Notes:**
- **Only update `en-GB/translation.toml`** - other locale files are managed separately
- **Only update `en-US/translation.toml`** - other locale files are managed separately
- Use descriptive keys that match your component's `t()` calls
- Include tooltip translations if you created tooltip hooks
- Add `options.*` keys if your tool has settings with descriptions
+1 -1
View File
@@ -426,7 +426,7 @@ The frontend is organized with a clear separation of concerns:
## Translation Rules
- **CRITICAL**: Always update translations in `en-GB` only, never `en-US`
- **CRITICAL**: Always update translations in `en-US` only - all other languages (including `en-GB`) are handled separately
- Translation files are located in `frontend/editor/public/locales/`
## Important Notes
+12 -1
View File
@@ -52,6 +52,17 @@ This guide focuses on developing for Stirling 2.0, including both the React fron
- Rust and Cargo (required for Tauri desktop app development)
- Tauri CLI (install with `cargo install tauri-cli`)
### Optional System Dependencies
These are not required to run the app but enable specific features. The app detects them at startup and disables the relevant features if they are missing.
| Dependency | Feature | Install |
|---|---|---|
| LibreOffice | File-to-PDF conversions | `brew install libreoffice` / `apt install libreoffice` |
| Tesseract | OCR | `brew install tesseract` / `apt install tesseract-ocr` |
| WeasyPrint | AI document creation | `brew install weasyprint` / `apt install weasyprint` |
| qpdf | PDF optimisation | `brew install qpdf` / `apt install qpdf` |
### Setup Steps
1. Clone the repository:
@@ -576,7 +587,7 @@ When adding a new feature or modifying existing ones in Stirling-PDF, you'll nee
Find the existing `messages.properties` files in the `stirling-pdf/src/main/resources` directory. You'll see files like:
- `messages.properties` (default, usually English)
- `messages_en_GB.properties`
- `messages_en_US.properties`
- `messages_fr_FR.properties`
- `messages_de_DE.properties`
- etc.
+12 -16
View File
@@ -60,24 +60,20 @@ tasks:
dev:saas:
desc: "Start SaaS backend + frontend concurrently on free ports"
vars:
PORTS:
sh: '{{if eq OS "windows"}}{{.FIND_FREE_PORT_PS}} 8080 5173{{else}}{{.FIND_FREE_PORT_SH}} 8080 5173{{end}}'
BACKEND_PORT: '{{index (splitList "\n" .PORTS) 0}}'
FRONTEND_PORT: '{{index (splitList "\n" .PORTS) 1}}'
deps:
- task: backend:dev:saas
vars:
PORT: '{{.BACKEND_PORT}}'
- task: frontend:dev:saas
vars:
PORT: '{{.FRONTEND_PORT}}'
BACKEND_URL: 'http://localhost:{{.BACKEND_PORT}}'
OPEN: "true"
cmds:
- task: dev:_all
vars: { FRONTEND: saas, BACKEND: saas }
dev:all:
desc: "Start backend + frontend + engine concurrently on free ports"
cmds:
- task: dev:_all
dev:_all:
internal: true
vars:
FRONTEND: '{{.FRONTEND | default "proprietary"}}'
BACKEND: '{{.BACKEND | default "proprietary"}}'
PORTS:
sh: '{{if eq OS "windows"}}{{.FIND_FREE_PORT_PS}} 8080 5173 5001{{else}}{{.FIND_FREE_PORT_SH}} 8080 5173 5001{{end}}'
BACKEND_PORT: '{{index (splitList "\n" .PORTS) 0}}'
@@ -87,11 +83,11 @@ tasks:
- task: engine:dev
vars:
PORT: '{{.ENGINE_PORT}}'
- task: backend:dev
- task: 'backend:dev:{{.BACKEND}}'
vars:
PORT: '{{.BACKEND_PORT}}'
AIENGINE_URL: 'http://localhost:{{.ENGINE_PORT}}'
- task: frontend:dev
- task: 'frontend:dev:{{.FRONTEND}}'
vars:
PORT: '{{.FRONTEND_PORT}}'
BACKEND_URL: 'http://localhost:{{.BACKEND_PORT}}'
@@ -1,73 +0,0 @@
package stirling.software.SPDF.pdf.parser;
import static stirling.software.SPDF.pdf.parser.PdfModels.*;
import java.io.IOException;
import java.util.List;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.springframework.context.annotation.Primary;
import org.springframework.stereotype.Service;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
/**
* Chains table parsers in priority order: Tabula lattice → Tabula stream → {@link
* LineAlignmentTableParser}. The first parser returning a result above {@link
* #TABULA_CONFIDENCE_THRESHOLD} wins; results from different parsers are never mixed on one page.
*/
@Service
@Primary
@RequiredArgsConstructor
@Slf4j
public class CompositeTableParser implements TableParser {
/** Min Tabula confidence to accept results; below this LineAlignment is tried instead. */
static final float TABULA_CONFIDENCE_THRESHOLD = 0.5f;
private final TabulaTableParser tabulaParser;
private final LineAlignmentTableParser lineAlignmentParser;
@Override
public List<TableFragment> parse(PDDocument document, RawPage rawPage) throws IOException {
// Step 1: Tabula lattice mode (ruled/bordered tables).
List<TableFragment> latticeResults = filterConfident(tabulaParser.parse(document, rawPage));
if (!latticeResults.isEmpty()) {
log.debug(
"Page {}: using Tabula lattice ({} table(s))",
rawPage.pageNumber(),
latticeResults.size());
return latticeResults;
}
// Step 2: Tabula stream mode (borderless/whitespace-delimited tables).
// parseStream is not on the TableParser interface — this intentionally couples to the
// concrete TabulaTableParser since stream mode is a Tabula-specific concept.
List<TableFragment> streamResults =
filterConfident(tabulaParser.parseStream(document, rawPage));
if (!streamResults.isEmpty()) {
log.debug(
"Page {}: using Tabula stream ({} table(s))",
rawPage.pageNumber(),
streamResults.size());
return streamResults;
}
// Step 3: Geometry-based line-alignment fallback.
List<TableFragment> lineResults = lineAlignmentParser.parse(document, rawPage);
if (!lineResults.isEmpty()) {
log.debug(
"Page {}: using LineAlignment ({} table(s))",
rawPage.pageNumber(),
lineResults.size());
return lineResults;
}
return List.of();
}
private List<TableFragment> filterConfident(List<TableFragment> tables) {
return tables.stream().filter(t -> t.confidence() >= TABULA_CONFIDENCE_THRESHOLD).toList();
}
}
@@ -1,528 +0,0 @@
package stirling.software.SPDF.pdf.parser;
import static stirling.software.SPDF.pdf.parser.PdfModels.*;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.Comparator;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.TreeMap;
import java.util.regex.Pattern;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.springframework.stereotype.Service;
import lombok.extern.slf4j.Slf4j;
/**
* Fallback {@link TableParser} for borderless financial tables using text geometry.
*
* <p>Identifies "anchor lines" (≥2 numeric tokens), builds a column grid from their right-edge
* positions, groups vertically proximate anchor lines into table candidates, then scores each group
* on column consistency and anchor density (confidence ceiling 0.85).
*/
@Service
@Slf4j
public class LineAlignmentTableParser implements TableParser {
/** Width in points of each column position bucket. */
static final float COLUMN_BUCKET_PT = 5f;
/** Tolerance in buckets when matching a token's right-edge to a confirmed column position. */
private static final int COLUMN_MATCH_BUCKETS = 2;
/** Maximum gap (as a multiple of modal line spacing) before splitting a group. */
private static final float MAX_GAP_FACTOR = 2.5f;
/** Minimum anchor rows (numeric-heavy) to form a valid table. */
static final int MIN_TABLE_ROWS = 3;
/** Minimum confirmed column positions to form a valid table. */
static final int MIN_COLUMNS = 2;
/**
* Min fraction of anchor lines a column must appear on to be confirmed (permissive for N/A
* rows).
*/
private static final double COLUMN_MIN_FREQUENCY = 0.40;
/**
* Matches financial numeric tokens: integers, decimals, parenthetical negatives, currency,
* percent, nil dashes.
*/
private static final Pattern NUMERIC =
Pattern.compile("^[\\(\\-\\$£€¥]?\\d[\\d,\\.]*[\\)%]?$|^[-–—]$");
/**
* Lines within this y-distance are merged into one row (restores rows split by LineBuilder's
* column-gap logic).
*/
static final float ROW_MERGE_TOLERANCE_PT = 2f;
// ── public API ───────────────────────────────────────────────────────────────────────────────
@Override
public List<TableFragment> parse(PDDocument document, RawPage rawPage) throws IOException {
List<RawLine> lines = rawPage.lines();
if (lines.size() < MIN_TABLE_ROWS) return List.of();
float modalSpacing = computeModalSpacing(lines);
List<TokenizedLine> tokenized =
mergeCoincidentLines(lines.stream().map(this::tokenize).toList());
List<TokenizedLine> anchors = tokenized.stream().filter(TokenizedLine::isAnchor).toList();
if (anchors.size() < MIN_TABLE_ROWS) return List.of();
List<Float> columnGrid = buildColumnGrid(anchors);
if (columnGrid.size() < MIN_COLUMNS) {
log.debug(
"Page {}: LineAlignment — fewer than {} confirmed columns, skipping",
rawPage.pageNumber(),
MIN_COLUMNS);
return List.of();
}
List<List<TokenizedLine>> groups = groupRows(tokenized, columnGrid, modalSpacing);
List<TableFragment> results = new ArrayList<>();
for (int i = 0; i < groups.size(); i++) {
buildFragment(groups.get(i), columnGrid, rawPage.pageNumber(), i)
.ifPresent(results::add);
}
log.debug(
"Page {}: LineAlignment detected {} table(s) ({} anchor lines, {} columns)",
rawPage.pageNumber(),
results.size(),
anchors.size(),
columnGrid.size());
return results;
}
// ── coincident-line merging ──────────────────────────────────────────────────────────────────
/**
* Merges tokenised lines sharing the same y-position into one row, rejoining label/value halves
* split by LineBuilder.
*/
List<TokenizedLine> mergeCoincidentLines(List<TokenizedLine> tokenized) {
if (tokenized.size() < 2) return tokenized;
List<TokenizedLine> result = new ArrayList<>();
int i = 0;
while (i < tokenized.size()) {
float baseY = tokenized.get(i).line().bounds().y();
int j = i + 1;
while (j < tokenized.size()
&& Math.abs(tokenized.get(j).line().bounds().y() - baseY)
<= ROW_MERGE_TOLERANCE_PT) {
j++;
}
if (j == i + 1) {
result.add(tokenized.get(i));
} else {
result.add(mergeGroup(tokenized.subList(i, j)));
}
i = j;
}
return result;
}
private TokenizedLine mergeGroup(List<TokenizedLine> group) {
List<TextFragment> mergedFragments =
group.stream()
.flatMap(tl -> tl.line().fragments().stream())
.sorted(Comparator.comparingDouble(f -> f.bounds().x()))
.toList();
Bounds mergedBounds =
group.stream()
.map(tl -> tl.line().bounds())
.reduce(Bounds::merge)
.orElse(group.get(0).line().bounds());
RawLine mergedLine =
new RawLine(
group.get(0).line().lineId(),
mergedFragments,
mergedBounds,
group.get(0).line().pageNumber());
return tokenize(mergedLine);
}
// ── tokenisation ─────────────────────────────────────────────────────────────────────────────
/**
* Splits fragments into word-level tokens; x-positions are estimated linearly within each
* fragment.
*/
TokenizedLine tokenize(RawLine line) {
List<LineToken> tokens = new ArrayList<>();
for (TextFragment frag : line.fragments()) {
tokens.addAll(tokensFromFragment(frag));
}
List<LineToken> numeric = tokens.stream().filter(LineToken::numeric).toList();
return new TokenizedLine(line, tokens, numeric);
}
private List<LineToken> tokensFromFragment(TextFragment frag) {
String raw = frag.text();
if (raw == null || raw.isBlank()) return List.of();
float fragX = frag.bounds().x();
float fragWidth = frag.bounds().width();
int rawLen = raw.length();
List<LineToken> result = new ArrayList<>();
int offset = 0;
for (String part : raw.split("\\s+")) {
if (part.isEmpty()) {
offset++;
continue;
}
int idx = raw.indexOf(part, offset);
if (idx < 0) idx = offset;
float tokenX = rawLen > 0 ? fragX + ((float) idx / rawLen) * fragWidth : fragX;
float tokenRight =
rawLen > 0
? fragX + ((float) (idx + part.length()) / rawLen) * fragWidth
: fragX + fragWidth;
result.add(new LineToken(part, tokenX, tokenRight, NUMERIC.matcher(part).matches()));
offset = idx + part.length();
}
return result;
}
// ── column grid ──────────────────────────────────────────────────────────────────────────────
/**
* Returns confirmed column right-edge positions — those appearing on ≥ {@value
* #COLUMN_MIN_FREQUENCY} × N anchor lines.
*/
private List<Float> buildColumnGrid(List<TokenizedLine> anchors) {
// bucket → set of line indices that contributed a numeric token to that bucket
Map<Integer, List<Integer>> bucketLines = new HashMap<>();
for (int i = 0; i < anchors.size(); i++) {
for (LineToken t : anchors.get(i).numeric()) {
int bucket = bucket(t.right());
bucketLines.computeIfAbsent(bucket, k -> new ArrayList<>()).add(i);
}
}
int minHits =
Math.max(MIN_TABLE_ROWS, (int) Math.ceil(anchors.size() * COLUMN_MIN_FREQUENCY));
// Confirmed buckets → average right-edge for that bucket
TreeMap<Integer, Float> confirmed = new TreeMap<>();
for (Map.Entry<Integer, List<Integer>> entry : bucketLines.entrySet()) {
// Count distinct lines
long distinctLines = entry.getValue().stream().distinct().count();
if (distinctLines >= minHits) {
double avg =
entry.getValue().stream()
.distinct() // weight each line equally regardless of token count
.mapToDouble(
lineIdx ->
avgRightEdgeForBucket(
anchors, lineIdx, entry.getKey()))
.average()
.orElse(entry.getKey() * (double) COLUMN_BUCKET_PT);
confirmed.put(entry.getKey(), (float) avg);
}
}
return new ArrayList<>(confirmed.values()); // already sorted by bucket (left to right)
}
/**
* Returns the average right-edge position of tokens in {@code line} whose bucket matches {@code
* targetBucket}, falling back to the bucket's nominal centre when no tokens match.
*/
private double avgRightEdgeForBucket(
List<TokenizedLine> anchors, int lineIdx, int targetBucket) {
return anchors.get(lineIdx).numeric().stream()
.filter(t -> bucket(t.right()) == targetBucket)
.mapToDouble(LineToken::right)
.average()
.orElse(targetBucket * (double) COLUMN_BUCKET_PT);
}
// ── grouping ─────────────────────────────────────────────────────────────────────────────────
/**
* Groups anchor lines into table candidates, including adjacent label rows; a gap &gt;
* MAX_GAP_FACTOR × modal spacing splits groups.
*/
private List<List<TokenizedLine>> groupRows(
List<TokenizedLine> all, List<Float> columnGrid, float modalSpacing) {
float maxGap = modalSpacing > 0 ? modalSpacing * MAX_GAP_FACTOR : 30f;
List<List<TokenizedLine>> groups = new ArrayList<>();
List<TokenizedLine> current = new ArrayList<>();
for (int i = 0; i < all.size(); i++) {
TokenizedLine tl = all.get(i);
boolean fits = tl.isAnchor() && matchesGrid(tl, columnGrid);
if (current.isEmpty()) {
if (fits) current.add(tl);
continue;
}
float gap =
tl.line().bounds().y()
- current.get(current.size() - 1).line().bounds().bottom();
if (gap > maxGap) {
groups.add(current);
current = new ArrayList<>();
if (fits) current.add(tl);
continue;
}
if (fits) {
current.add(tl);
} else if (!tl.line().text().isBlank()) {
// Include non-anchor lines (labels) only if they have text and are within
// proximity.
current.add(tl);
}
}
if (!current.isEmpty()) groups.add(current);
return groups.stream().filter(g -> hasEnoughAnchorRows(g, columnGrid)).toList();
}
private boolean hasEnoughAnchorRows(List<TokenizedLine> group, List<Float> columnGrid) {
return group.stream().filter(r -> r.isAnchor() && matchesGrid(r, columnGrid)).count()
>= MIN_TABLE_ROWS;
}
/** A line "matches" the grid when ≥ 60 % of its numeric tokens land in confirmed columns. */
private boolean matchesGrid(TokenizedLine tl, List<Float> columnGrid) {
if (tl.numeric().isEmpty()) return false;
long matches =
tl.numeric().stream()
.filter(t -> nearestColumnIndex(t.right(), columnGrid) >= 0)
.count();
return (double) matches / tl.numeric().size() >= 0.60;
}
private boolean hasInconsistentColumnMatch(TokenizedLine tl, List<Float> columnGrid) {
if (tl.numeric().isEmpty()) return false;
long hits =
tl.numeric().stream()
.filter(t -> nearestColumnIndex(t.right(), columnGrid) >= 0)
.count();
return (double) hits / tl.numeric().size() < 0.60;
}
// ── fragment assembly ────────────────────────────────────────────────────────────────────────
private Optional<TableFragment> buildFragment(
List<TokenizedLine> group, List<Float> columnGrid, int pageNumber, int tableIndex) {
long anchorCount =
group.stream().filter(r -> r.isAnchor() && matchesGrid(r, columnGrid)).count();
if (anchorCount < MIN_TABLE_ROWS) return Optional.empty();
List<String> warnings = new ArrayList<>();
List<List<String>> rawRows = new ArrayList<>();
List<TableRow> rows = new ArrayList<>();
for (int rowIdx = 0; rowIdx < group.size(); rowIdx++) {
TokenizedLine tl = group.get(rowIdx);
List<String> rawRow = buildRawRow(tl, columnGrid);
rawRows.add(Collections.unmodifiableList(rawRow));
rows.add(buildTableRow(rowIdx, tl, rawRow, columnGrid));
}
// Column count = 1 label column + confirmed numeric columns
int colCount = columnGrid.size() + 1;
Bounds bounds = computeGroupBounds(group);
float confidence = computeConfidence(group, columnGrid, warnings);
return Optional.of(
new TableFragment(
"tbl-la-p" + pageNumber + "-" + tableIndex,
pageNumber,
bounds,
List.of(),
Collections.unmodifiableList(rows),
Collections.unmodifiableList(rawRows),
colCount,
confidence,
Collections.unmodifiableList(warnings),
null));
}
/**
* Builds a raw row as a list of strings: index 0 = label text, indices 1..N = column values.
*/
private List<String> buildRawRow(TokenizedLine tl, List<Float> columnGrid) {
String[] cells = new String[columnGrid.size() + 1];
Arrays.fill(cells, "");
// Separate label tokens (those not landing in any confirmed column) from column tokens.
List<String> labelParts = new ArrayList<>();
for (LineToken token : tl.all()) {
int col = nearestColumnIndex(token.right(), columnGrid);
if (col >= 0 && token.numeric()) {
int cellIdx = col + 1;
cells[cellIdx] =
cells[cellIdx].isEmpty()
? token.text()
: cells[cellIdx] + " " + token.text();
} else {
labelParts.add(token.text());
}
}
cells[0] = String.join(" ", labelParts).trim();
return Arrays.asList(cells);
}
private TableRow buildTableRow(
int rowIdx, TokenizedLine tl, List<String> rawRow, List<Float> columnGrid) {
List<TableCell> cells = new ArrayList<>(rawRow.size());
// Label cell: use the line's full bounds as an approximation.
cells.add(TableCell.of(0, rawRow.get(0), tl.line().bounds()));
for (int col = 0; col < columnGrid.size(); col++) {
String text = col + 1 < rawRow.size() ? rawRow.get(col + 1) : "";
float right = columnGrid.get(col);
float left = col > 0 ? columnGrid.get(col - 1) : right - 50f;
Bounds cellBounds =
new Bounds(
left,
tl.line().bounds().y(),
right - left,
tl.line().bounds().height());
cells.add(TableCell.of(col + 1, text, cellBounds));
}
return new TableRow(rowIdx, Collections.unmodifiableList(cells));
}
// ── confidence scoring ───────────────────────────────────────────────────────────────────────
/**
* Heuristic score in [0.0, 0.85] (ceiling keeps results below Tabula lattice which starts at
* 1.0). Base 0.70; +0.05/col beyond 2 (max +0.10); +0.05 at ≥5 anchors, +0.05 at ≥8; 0.15 if
* &gt;30 % of anchors have inconsistent columns; 0.10 if non-anchors outnumber anchors.
*/
private float computeConfidence(
List<TokenizedLine> group, List<Float> columnGrid, List<String> warnings) {
float score = 0.70f;
long anchorCount =
group.stream().filter(r -> r.isAnchor() && matchesGrid(r, columnGrid)).count();
long totalRows = group.size();
// More columns
int extraCols = Math.min(columnGrid.size() - MIN_COLUMNS, 2);
score += extraCols * 0.05f;
// More anchor rows
if (anchorCount >= 5) score += 0.05f;
if (anchorCount >= 8) score += 0.05f;
// Inconsistent column matching
long inconsistent =
group.stream()
.filter(TokenizedLine::isAnchor)
.filter(tl -> hasInconsistentColumnMatch(tl, columnGrid))
.count();
if (inconsistent > anchorCount * 0.30) {
score -= 0.15f;
warnings.add(
"Column match inconsistent on "
+ inconsistent
+ "/"
+ anchorCount
+ " anchor rows");
}
// Label-heavy
long nonAnchor = totalRows - anchorCount;
if (nonAnchor > anchorCount) {
score -= 0.10f;
warnings.add(
"Non-anchor rows ("
+ nonAnchor
+ ") outnumber anchor rows ("
+ anchorCount
+ ")");
}
return Math.max(0f, Math.min(0.85f, score));
}
// ── utility ──────────────────────────────────────────────────────────────────────────────────
/**
* Returns the grid index nearest to {@code rightEdge}, or -1 if none is within {@value
* #COLUMN_MATCH_BUCKETS} buckets.
*/
private int nearestColumnIndex(float rightEdge, List<Float> grid) {
int nearest = -1;
float minDist = COLUMN_MATCH_BUCKETS * COLUMN_BUCKET_PT + 1f;
for (int i = 0; i < grid.size(); i++) {
float dist = Math.abs(rightEdge - grid.get(i));
if (dist < minDist) {
minDist = dist;
nearest = i;
}
}
return nearest;
}
private Bounds computeGroupBounds(List<TokenizedLine> group) {
return group.stream()
.map(tl -> tl.line().bounds())
.reduce(Bounds::merge)
.orElse(new Bounds(0, 0, 0, 0));
}
/** Modal gap between consecutive line edges, used to calibrate the group-split threshold. */
private float computeModalSpacing(List<RawLine> lines) {
if (lines.size() < 2) return 0f;
Map<Float, Long> freq = new HashMap<>();
for (int i = 1; i < lines.size(); i++) {
float gap = lines.get(i).bounds().y() - lines.get(i - 1).bounds().bottom();
if (gap > 0) freq.merge(Math.round(gap / 2f) * 2f, 1L, Long::sum);
}
return freq.entrySet().stream()
.max(Map.Entry.comparingByValue())
.map(Map.Entry::getKey)
.orElse(0f);
}
private static int bucket(float x) {
return Math.round(x / COLUMN_BUCKET_PT);
}
// ── private data types ───────────────────────────────────────────────────────────────────────
/** A word-level token with an approximate right-edge x-position. */
record LineToken(String text, float x, float right, boolean numeric) {}
/** A {@link RawLine} with tokens pre-computed; an "anchor" has ≥ 2 numeric tokens. */
record TokenizedLine(RawLine line, List<LineToken> all, List<LineToken> numeric) {
boolean isAnchor() {
return numeric.size() >= 2;
}
}
}
@@ -1,139 +0,0 @@
package stirling.software.SPDF.pdf.parser;
import static stirling.software.SPDF.pdf.parser.PdfModels.*;
import java.util.ArrayList;
import java.util.Comparator;
import java.util.List;
import org.springframework.stereotype.Service;
import lombok.extern.slf4j.Slf4j;
/**
* Groups {@link TextFragment} objects into visual {@link RawLine}s using baseline proximity.
*
* <p>Fragments are on the same line when their baselines are within a font-size-derived tolerance.
* A new line starts whenever the horizontal gap exceeds an adaptive column-gap threshold ({@code
* max(effectiveWidth * COLUMN_GAP_RATIO, COLUMN_GAP_MIN_PT)}), splitting two-column text.
*/
@Service
@Slf4j
public class LineBuilder {
/** Baseline tolerance as a fraction of font size; 0.5 keeps mixed-size text on one line. */
private static final float BASELINE_TOLERANCE_FACTOR = 0.5f;
/** Absolute minimum tolerance so tiny font sizes don't collapse multi-line content. */
private static final float MIN_BASELINE_TOLERANCE = 2f;
/**
* Column-gap threshold as a fraction of page width; 0.10 clears tab stops but stays below
* two-column gutters.
*/
static final float COLUMN_GAP_RATIO = 0.10f;
/** Floor for the column-gap threshold so narrow pages don't over-split lines. */
static final float COLUMN_GAP_MIN_PT = 40f;
public List<RawLine> build(List<TextFragment> fragments, int pageNumber) {
if (fragments.isEmpty()) return List.of();
float effectiveWidth = inferEffectiveWidth(fragments);
float columnGapThreshold = Math.max(effectiveWidth * COLUMN_GAP_RATIO, COLUMN_GAP_MIN_PT);
log.debug(
"LineBuilder page {}: effectiveWidth={:.1f}pt, columnGapThreshold={:.1f}pt",
pageNumber,
effectiveWidth,
columnGapThreshold);
// Sort top-to-bottom first, then left-to-right within the same baseline band.
List<TextFragment> sorted =
fragments.stream()
.sorted(
Comparator.comparingDouble(TextFragment::baseline)
.thenComparingDouble(f -> f.bounds().x()))
.toList();
List<List<TextFragment>> groups = groupByBaseline(sorted, columnGapThreshold);
List<RawLine> lines = new ArrayList<>(groups.size());
for (int i = 0; i < groups.size(); i++) {
List<TextFragment> group =
groups.get(i).stream()
.sorted(Comparator.comparingDouble(f -> f.bounds().x()))
.toList();
Bounds lineBounds =
group.stream()
.map(TextFragment::bounds)
.reduce(Bounds::merge)
.orElse(new Bounds(0, 0, 0, 0));
lines.add(new RawLine("ln-p" + pageNumber + "-" + i, group, lineBounds, pageNumber));
}
return lines;
}
private List<List<TextFragment>> groupByBaseline(
List<TextFragment> sorted, float columnGapThreshold) {
List<List<TextFragment>> groups = new ArrayList<>();
List<TextFragment> current = new ArrayList<>();
float currentBaseline = Float.NaN;
for (TextFragment fragment : sorted) {
if (current.isEmpty()) {
current.add(fragment);
currentBaseline = fragment.baseline();
continue;
}
float maxFontSize =
Math.max(
fragment.fontSize(),
(float)
current.stream()
.mapToDouble(TextFragment::fontSize)
.max()
.orElse(0));
float tolerance =
Math.max(maxFontSize * BASELINE_TOLERANCE_FACTOR, MIN_BASELINE_TOLERANCE);
boolean sameBaseline = Math.abs(fragment.baseline() - currentBaseline) <= tolerance;
boolean columnGap = sameBaseline && hasColumnGap(fragment, current, columnGapThreshold);
if (sameBaseline && !columnGap) {
current.add(fragment);
// Anchor to the weighted mean baseline so long lines stay stable.
currentBaseline =
(currentBaseline * (current.size() - 1) + fragment.baseline())
/ current.size();
} else {
groups.add(current);
current = new ArrayList<>();
current.add(fragment);
currentBaseline = fragment.baseline();
}
}
if (!current.isEmpty()) groups.add(current);
return groups;
}
/**
* True when the gap from the rightmost fragment in {@code group} to {@code next} exceeds {@code
* threshold}.
*/
private static boolean hasColumnGap(
TextFragment next, List<TextFragment> group, float threshold) {
float lastRight = group.get(group.size() - 1).bounds().right();
return next.bounds().x() - lastRight > threshold;
}
/** Infers effective page width from the rightmost fragment right-edge plus a 10 % margin. */
private static float inferEffectiveWidth(List<TextFragment> fragments) {
double maxRight =
fragments.stream().mapToDouble(f -> f.bounds().right()).max().orElse(500.0);
return (float) maxRight * 1.10f;
}
}
@@ -1,79 +0,0 @@
package stirling.software.SPDF.pdf.parser;
import static stirling.software.SPDF.pdf.parser.PdfModels.*;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDPage;
import org.apache.pdfbox.pdmodel.common.PDRectangle;
import org.springframework.stereotype.Service;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
/**
* Runs the per-page ingestion pipeline: {@link WordExtractingStripper} → {@link LineBuilder} →
* {@link TableParser}, producing a {@link PdfModels.ParsedPage} per page. The caller owns the
* {@link PDDocument} lifecycle.
*/
@Service
@RequiredArgsConstructor
@Slf4j
public class PdfIngester {
private final LineBuilder lineBuilder;
private final TableParser tableParser;
public List<ParsedPage> parse(PDDocument document) throws IOException {
return parse(document, document.getNumberOfPages());
}
public List<ParsedPage> parse(PDDocument document, int maxPages) throws IOException {
int pageCount = Math.min(document.getNumberOfPages(), maxPages);
List<ParsedPage> pages = new ArrayList<>(pageCount);
long fragmentsMs = 0;
long tablesMs = 0;
long t0 = System.currentTimeMillis();
for (int p = 1; p <= pageCount; p++) {
long ft = System.currentTimeMillis();
List<TextFragment> fragments = extractFragments(document, p);
fragmentsMs += System.currentTimeMillis() - ft;
PDPage page = document.getPage(p - 1);
PDRectangle mediaBox = page.getMediaBox();
List<RawLine> lines = lineBuilder.build(fragments, p);
RawPage rawPage = new RawPage(p, mediaBox.getWidth(), mediaBox.getHeight(), lines);
long tt = System.currentTimeMillis();
List<TableFragment> tables = tableParser.parse(document, rawPage);
tablesMs += System.currentTimeMillis() - tt;
log.debug(
"Page {}: {} fragments → {} lines, {} table(s)",
p,
fragments.size(),
lines.size(),
tables.size());
pages.add(new ParsedPage(p, mediaBox.getWidth(), mediaBox.getHeight(), tables, lines));
}
log.info(
"[timing] parse pages={} total={}ms fragments={}ms tables={}ms",
pageCount,
System.currentTimeMillis() - t0,
fragmentsMs,
tablesMs);
return pages;
}
private List<TextFragment> extractFragments(PDDocument document, int pageNumber)
throws IOException {
WordExtractingStripper stripper = new WordExtractingStripper(pageNumber);
stripper.getText(document);
return stripper.getFragments();
}
}
@@ -1,113 +0,0 @@
package stirling.software.SPDF.pdf.parser;
import static stirling.software.SPDF.pdf.parser.PdfModels.*;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import org.apache.pdfbox.pdmodel.PDPage;
import org.apache.pdfbox.pdmodel.font.PDFont;
import org.apache.pdfbox.text.PDFTextStripper;
import org.apache.pdfbox.text.TextPosition;
/**
* Extends {@link PDFTextStripper} to capture per-fragment geometry and font metadata.
*
* <p>Overrides {@link #writeString} to split each content-stream string into word-level {@link
* TextFragment}s with bounding boxes, baseline, font name, and bold flag. Coordinates are in
* PDFTextStripper space: (0,0) top-left, Y increases downward, {@code getY()} is the baseline.
*/
class WordExtractingStripper extends PDFTextStripper {
private final int targetPage;
private final List<TextFragment> fragments = new ArrayList<>();
private int fragmentIndex = 0;
WordExtractingStripper(int pageNumber) throws IOException {
this.targetPage = pageNumber;
setStartPage(pageNumber);
setEndPage(pageNumber);
setSortByPosition(true);
}
@Override
protected void startPage(PDPage page) throws IOException {
super.startPage(page);
fragments.clear();
fragmentIndex = 0;
}
@Override
protected void writeString(String text, List<TextPosition> textPositions) throws IOException {
if (text == null || text.isBlank()) return;
// Fast path: no whitespace → emit one fragment (most financial PDFs have each
// number as its own string operation, so this is the common case).
if (text.indexOf(' ') < 0) {
emitFragment(text, textPositions);
return;
}
// Per-word splitting requires 1:1 text-char to TextPosition correspondence.
// Fall back to one fragment when sizes differ (ligatures, encoding edge cases).
if (textPositions.size() != text.length()) {
emitFragment(text, textPositions);
return;
}
// Emit one TextFragment per whitespace-delimited word with accurate per-word bounds.
int start = 0;
for (int i = 0; i <= text.length(); i++) {
if (i == text.length() || text.charAt(i) == ' ') {
if (start < i) {
emitFragment(text.substring(start, i), textPositions.subList(start, i));
}
start = i + 1;
}
}
}
private void emitFragment(String text, List<TextPosition> positions) {
if (positions.isEmpty()) return;
float minX = Float.MAX_VALUE;
float minY = Float.MAX_VALUE;
float maxRight = -Float.MAX_VALUE;
float maxBaseline = -Float.MAX_VALUE;
TextPosition first = null;
for (TextPosition tp : positions) {
if (tp == null) continue;
if (first == null) first = tp;
float x = tp.getX();
// getY() is the baseline; top of character = getY() - getHeight().
float top = tp.getY() - tp.getHeight();
float right = x + tp.getWidth();
float baseline = tp.getY();
minX = Math.min(minX, x);
minY = Math.min(minY, top);
maxRight = Math.max(maxRight, right);
maxBaseline = Math.max(maxBaseline, baseline);
}
if (first == null) return;
PDFont font = first.getFont();
String fontName = font != null ? font.getName() : "";
boolean bold = fontName != null && fontName.toLowerCase().contains("bold");
// getHeight() gives the rendered glyph height, which is the most reliable visual size.
float fontSize = first.getHeight();
Bounds bounds = new Bounds(minX, minY, maxRight - minX, maxBaseline - minY);
String id = "tf-p" + targetPage + "-" + fragmentIndex++;
fragments.add(new TextFragment(id, text, bounds, maxBaseline, fontSize, fontName, bold));
}
List<TextFragment> getFragments() {
return Collections.unmodifiableList(fragments);
}
}
@@ -11,23 +11,39 @@ public interface FileStore {
/** Stored file record. */
record Stored(String fileId, long size) {}
/** Store the given stream and return a generated file id and total bytes written. */
Stored store(InputStream in, String originalName) throws IOException;
/**
* Store the given stream and return a generated file id and total bytes written. {@code owner}
* may be null to indicate the file has no associated user (anonymous / desktop / async job with
* no propagated security context); a non-null value is persisted alongside the data so {@link
* #getOwner(String)} can return it later for authorization checks.
*/
Stored store(InputStream in, String originalName, String owner) throws IOException;
/** Store with no owner. Equivalent to {@link #store(InputStream, String, String)} with null. */
default Stored store(InputStream in, String originalName) throws IOException {
return store(in, originalName, null);
}
/**
* Store the file at {@code source} and return a generated file id and total bytes written.
*
* <p>Default implementation opens {@code source} as a stream and delegates to {@link
* #store(InputStream, String)}. Local-disk implementations should override to use a direct
* file-to-file copy ({@code Files.copy(source, dest)} can use {@code sendfile(2)} on Linux),
* which avoids the two-memory-copy hit of streaming a disk-backed upload through the JVM heap.
* #store(InputStream, String, String)}. Local-disk implementations should override to use a
* direct file-to-file copy ({@code Files.copy(source, dest)} can use {@code sendfile(2)} on
* Linux), which avoids the two-memory-copy hit of streaming a disk-backed upload through the
* JVM heap.
*/
default Stored store(Path source, String originalName) throws IOException {
default Stored store(Path source, String originalName, String owner) throws IOException {
try (InputStream in = Files.newInputStream(source)) {
return store(in, originalName);
return store(in, originalName, owner);
}
}
/** Store with no owner. Equivalent to {@link #store(Path, String, String)} with null. */
default Stored store(Path source, String originalName) throws IOException {
return store(source, originalName, null);
}
/** Open the stored file for streaming reads. Caller closes. */
InputStream retrieve(String fileId) throws IOException;
@@ -42,4 +58,12 @@ public interface FileStore {
/** Whether the file id exists in the store. */
boolean exists(String fileId);
/**
* Returns the owner identifier recorded at store time, or {@code null} if the file does not
* exist or was stored without an owner. Implementations must not throw when the file is missing
* or when the owner record is absent; they should return null so callers can treat "no owner"
* as a non-authoritative case.
*/
String getOwner(String fileId) throws IOException;
}
@@ -3,9 +3,12 @@ package stirling.software.common.cluster.inprocess;
import java.io.BufferedInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.UUID;
import java.util.concurrent.locks.ReentrantLock;
import java.util.regex.Pattern;
import lombok.extern.slf4j.Slf4j;
@@ -15,33 +18,47 @@ import stirling.software.common.cluster.FileStore;
@Slf4j
public class LocalDiskFileStore implements FileStore {
private static final String OWNER_SUFFIX = ".owner";
// File ids are generated as random UUIDs; reject anything else so a tainted id can never reach
// Files.* APIs (defence in depth on top of the resolve() prefix check, and silences CodeQL's
// path-injection finding on the resolveOwner sidecar lookup).
private static final Pattern UUID_PATTERN =
Pattern.compile(
"^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$");
private final String baseDirPath;
// Fixed-size lock stripes so concurrent store/delete on the same (or colliding) fileId
// serialise the data-file + owner-sidecar pair as one critical section. Striped (not
// per-id) so the map never has to be cleaned up; collisions across unrelated ids are
// harmless contention.
private static final int LOCK_STRIPES = 64;
private final ReentrantLock[] stripes = new ReentrantLock[LOCK_STRIPES];
public LocalDiskFileStore(String baseDirPath) {
this.baseDirPath = baseDirPath;
for (int i = 0; i < LOCK_STRIPES; i++) {
stripes[i] = new ReentrantLock();
}
}
@Override
public Stored store(InputStream in, String originalName) throws IOException {
public Stored store(InputStream in, String originalName, String owner) throws IOException {
String fileId = UUID.randomUUID().toString();
Path filePath = resolve(fileId);
Files.createDirectories(filePath.getParent());
ReentrantLock lock = acquire(fileId);
boolean success = false;
try {
long size = Files.copy(in, filePath);
writeOwner(fileId, owner);
success = true;
return new Stored(fileId, size);
} finally {
if (!success) {
try {
Files.deleteIfExists(filePath);
} catch (IOException cleanupEx) {
log.warn(
"Failed to clean up partial file {} after store failure",
filePath,
cleanupEx);
}
cleanupAfterFailedStore(fileId, filePath);
}
release(fileId, lock);
}
}
@@ -52,27 +69,44 @@ public class LocalDiskFileStore implements FileStore {
* the source size before copying so the post-copy stat is unnecessary.
*/
@Override
public Stored store(Path source, String originalName) throws IOException {
public Stored store(Path source, String originalName, String owner) throws IOException {
String fileId = UUID.randomUUID().toString();
Path filePath = resolve(fileId);
Files.createDirectories(filePath.getParent());
long size = Files.size(source);
ReentrantLock lock = acquire(fileId);
boolean success = false;
try {
Files.copy(source, filePath);
writeOwner(fileId, owner);
success = true;
return new Stored(fileId, size);
} finally {
if (!success) {
try {
Files.deleteIfExists(filePath);
} catch (IOException cleanupEx) {
log.warn(
"Failed to clean up partial file {} after store failure",
filePath,
cleanupEx);
}
cleanupAfterFailedStore(fileId, filePath);
}
release(fileId, lock);
}
}
private void writeOwner(String fileId, String owner) throws IOException {
if (owner == null || owner.isBlank()) {
return;
}
Path ownerPath = resolveOwner(fileId);
Files.write(ownerPath, owner.getBytes(StandardCharsets.UTF_8));
}
private void cleanupAfterFailedStore(String fileId, Path filePath) {
try {
Files.deleteIfExists(filePath);
} catch (IOException cleanupEx) {
log.warn("Failed to clean up partial file {} after store failure", filePath, cleanupEx);
}
try {
Files.deleteIfExists(resolveOwner(fileId));
} catch (IOException cleanupEx) {
log.warn("Failed to clean up owner sidecar for {} after store failure", fileId);
}
}
@@ -101,11 +135,26 @@ public class LocalDiskFileStore implements FileStore {
@Override
public boolean delete(String fileId) {
ReentrantLock lock = acquire(fileId);
try {
return Files.deleteIfExists(resolve(fileId));
} catch (IOException e) {
log.error("Error deleting file with ID: {}", fileId, e);
return false;
// Data first, owner second: a concurrent retrieve that observes the transient
// (data-gone, owner-still-present) window simply fails with IOException; the inverse
// order would briefly look like an unowned file and could grant cross-user access.
boolean removed;
try {
removed = Files.deleteIfExists(resolve(fileId));
} catch (IOException e) {
log.error("Error deleting file with ID: {}", fileId, e);
return false;
}
try {
Files.deleteIfExists(resolveOwner(fileId));
} catch (IOException e) {
log.warn("Error deleting owner sidecar for file ID: {}", fileId, e);
}
return removed;
} finally {
release(fileId, lock);
}
}
@@ -114,8 +163,21 @@ public class LocalDiskFileStore implements FileStore {
return Files.exists(resolve(fileId));
}
@Override
public String getOwner(String fileId) throws IOException {
Path ownerPath = resolveOwner(fileId);
if (!Files.exists(ownerPath)) {
return null;
}
byte[] bytes = Files.readAllBytes(ownerPath);
if (bytes.length == 0) {
return null;
}
return new String(bytes, StandardCharsets.UTF_8);
}
public Path resolve(String fileId) {
if (fileId.contains("..") || fileId.contains("/") || fileId.contains("\\")) {
if (fileId == null || !UUID_PATTERN.matcher(fileId).matches()) {
throw new IllegalArgumentException("Invalid file ID");
}
Path basePath = Path.of(baseDirPath).normalize().toAbsolutePath();
@@ -125,4 +187,19 @@ public class LocalDiskFileStore implements FileStore {
}
return resolvedPath;
}
private Path resolveOwner(String fileId) {
Path data = resolve(fileId);
return data.resolveSibling(data.getFileName().toString() + OWNER_SUFFIX);
}
private ReentrantLock acquire(String fileId) {
ReentrantLock lock = stripes[(fileId.hashCode() & Integer.MAX_VALUE) % LOCK_STRIPES];
lock.lock();
return lock;
}
private void release(String fileId, ReentrantLock lock) {
lock.unlock();
}
}
@@ -3,7 +3,6 @@ package stirling.software.common.configuration;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.List;
import java.util.Locale;
import java.util.Properties;
@@ -122,17 +121,17 @@ public class AppConfig {
@Bean(name = "RunningInDocker")
public boolean runningInDocker() {
return Files.exists(Paths.get("/.dockerenv"));
return Files.exists(Path.of("/.dockerenv"));
}
@Bean(name = "configDirMounted")
public boolean isRunningInDockerWithConfig() {
Path dockerEnv = Paths.get("/.dockerenv");
Path dockerEnv = Path.of("/.dockerenv");
// default to true if not docker
if (!Files.exists(dockerEnv)) {
return true;
}
Path mountInfo = Paths.get("/proc/1/mountinfo");
Path mountInfo = Path.of("/proc/1/mountinfo");
// this should always exist, if not some unknown usecase
if (!Files.exists(mountInfo)) {
return true;
@@ -7,7 +7,6 @@ import java.net.URISyntaxException;
import java.net.URL;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.StandardCopyOption;
import java.util.List;
@@ -27,7 +26,7 @@ public class ConfigInitializer {
public void ensureConfigExists() throws IOException, URISyntaxException {
// 1) If settings file doesn't exist, create from template
Path destPath = Paths.get(InstallationPathConfig.getSettingsPath());
Path destPath = Path.of(InstallationPathConfig.getSettingsPath());
boolean settingsFileExists = Files.exists(destPath);
@@ -39,7 +38,7 @@ public class ConfigInitializer {
if (settingsFileExists) {
// move settings.yml to settings.yml.{timestamp}.bak
Path backupPath =
Paths.get(
Path.of(
InstallationPathConfig.getSettingsPath()
+ "."
+ System.currentTimeMillis()
@@ -96,7 +95,7 @@ public class ConfigInitializer {
}
// 3) Ensure custom settings file exists
Path customSettingsPath = Paths.get(InstallationPathConfig.getCustomSettingsPath());
Path customSettingsPath = Path.of(InstallationPathConfig.getCustomSettingsPath());
if (Files.notExists(customSettingsPath)) {
Files.createFile(customSettingsPath);
log.info("Created custom_settings file: {}", customSettingsPath);
@@ -3,7 +3,6 @@ package stirling.software.common.configuration;
import java.nio.file.Files;
import java.nio.file.InvalidPathException;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.Collections;
import java.util.LinkedHashSet;
@@ -201,7 +200,7 @@ public class RuntimePathConfig {
try {
// Normalize to absolute path
Path path = Paths.get(pathStr.trim()).toAbsolutePath().normalize();
Path path = Path.of(pathStr.trim()).toAbsolutePath().normalize();
String normalizedPath = path.toString();
// Check for duplicates
@@ -224,9 +223,9 @@ public class RuntimePathConfig {
private void detectOverlappingPaths(List<String> paths) {
for (int i = 0; i < paths.size(); i++) {
Path path1 = Paths.get(paths.get(i));
Path path1 = Path.of(paths.get(i));
for (int j = i + 1; j < paths.size(); j++) {
Path path2 = Paths.get(paths.get(j));
Path path2 = Path.of(paths.get(j));
// Check if one path is a parent of the other
if (path1.startsWith(path2)) {
@@ -246,10 +245,10 @@ public class RuntimePathConfig {
private void validatePipelinePaths() {
try {
Path finishedPath = Paths.get(pipelineFinishedFoldersPath).toAbsolutePath().normalize();
Path finishedPath = Path.of(pipelineFinishedFoldersPath).toAbsolutePath().normalize();
for (String watchedPathStr : pipelineWatchedFoldersPaths) {
Path watchedPath = Paths.get(watchedPathStr).toAbsolutePath().normalize();
Path watchedPath = Path.of(watchedPathStr).toAbsolutePath().normalize();
// Check if watched folder is same as finished folder
if (watchedPath.equals(finishedPath)) {
@@ -77,8 +77,10 @@ public class ApplicationProperties {
private ProcessExecutor processExecutor = new ProcessExecutor();
private PdfEditor pdfEditor = new PdfEditor();
private AiEngine aiEngine = new AiEngine();
private Mcp mcp = new Mcp();
private InternalApi internalApi = new InternalApi();
private Cluster cluster = new Cluster();
private Policies policies = new Policies();
@Bean
public PropertySource<?> dynamicYamlPropertySource(ConfigurableEnvironment environment)
@@ -202,6 +204,45 @@ public class ApplicationProperties {
}
}
@Data
public static class Policies {
/**
* Absolute directories that policy folder input sources and output sinks may read from or
* write to. Empty (the default) disables folder access entirely, so a policy can never be
* pointed at an arbitrary server path. Stirling's own config directory is always
* off-limits, and folder access is always disabled in SaaS mode regardless of this list.
*/
private List<String> allowedFolderRoots = new java.util.ArrayList<>();
/** How often (seconds) the schedule trigger checks for policies whose schedule is due. */
private long scheduleSweepSeconds = 60;
/**
* How often (seconds) the folder-watch trigger reconciles its watch registrations and
* re-runs every folder-watch policy as a safety net for filesystem events that were missed
* (NFS, bind mounts, inotify-queue overflow).
*/
private long watchReconcileSeconds = 300;
/**
* How long (milliseconds) the folder-watch trigger keeps draining filesystem events after
* the first, so a burst from a single file copy coalesces into one run instead of many.
*/
private long watchQuietPeriodMs = 500;
/**
* SSE emitter timeout (milliseconds) for streamed runs; generous for long multi-step runs.
*/
private long streamTimeoutMs = 1800000;
/**
* How long (minutes) a finished run's in-memory state is retained before eviction,
* mirroring the job-result expiry so rich run state does not outlive the process. Active
* and paused runs are kept regardless of age.
*/
private int runExpiryMinutes = 30;
}
@Data
public static class PdfEditor {
private Cache cache = new Cache();
@@ -256,6 +297,103 @@ public class ApplicationProperties {
private int longRunningTimeoutSeconds = 600;
}
/**
* Model Context Protocol (MCP) server configuration. All keys live under the top-level {@code
* mcp.*} prefix. {@link #enabled} defaults to {@code false}: when off, no MCP beans are wired,
* no /mcp endpoint exists, and no protected-resource metadata is published.
*/
@Data
public static class Mcp {
/** Master switch. When {@code false} (default), no MCP beans are wired. */
private boolean enabled = false;
/**
* When {@code true} (default), invocations require an OAuth scope: {@code mcp.tools.read}
* for read-style operations and {@code mcp.tools.write} for write/destructive ones. When
* {@code false}, scope checks are skipped (use only if your IdP issues a single coarse
* scope).
*/
private boolean scopesEnabled = true;
/** How often to refresh the AI capabilities manifest from the engine. */
private int engineCapabilityRefreshMinutes = 5;
/**
* Tool allow-list (operation ids, e.g. {@code compress-pdf}). When non-empty, ONLY these
* operations are exposed over MCP; everything else is hidden, undescribable, and
* uninvocable - on top of the global endpoint enable/disable config. Empty = allow all.
*/
private List<String> allowedOperations = new ArrayList<>();
/**
* Tool deny-list (operation ids). Any operation listed here is removed from MCP even if it
* would otherwise be allowed. Applied after {@link #allowedOperations}.
*/
private List<String> blockedOperations = new ArrayList<>();
/** Max MCP request body size in bytes; inline file uploads ride in the JSON-RPC body. */
private long maxRequestBytes = 10L * 1024 * 1024;
/** Results up to this size return inline as base64; larger ones return a fileId only. */
private long maxInlineResponseBytes = 10L * 1024 * 1024;
private Auth auth = new Auth();
@Data
public static class Auth {
/**
* Authentication mode for the MCP endpoint. {@code oauth} (default) runs a full OAuth2
* resource server (JWT, RFC 8707 audience, RFC 9728 metadata). {@code apikey} accepts a
* Stirling per-user API key via the {@code X-API-KEY} header (or {@code Authorization:
* Bearer <key>}) and binds the request to that user - the low-friction self-host path,
* no external IdP required.
*/
private String mode = "oauth";
/** OAuth2 issuer URI, e.g. {@code http://localhost:9000}. Required when MCP is on. */
private String issuerUri = "";
/**
* JWKS URI. When blank, derived from the issuer's {@code
* /.well-known/openid-configuration} document.
*/
private String jwksUri = "";
/**
* RFC 8707 resource identifier of THIS MCP server, e.g. {@code
* http://localhost:8080/mcp}. Tokens that do not list this id in their {@code aud}
* claim are rejected with HTTP 401.
*/
private String resourceId = "";
/**
* Additional JWT audiences accepted at the MCP endpoint, on top of {@link #resourceId}.
* Empty (default) keeps strict RFC 8707 binding. Some IdPs cannot mint
* resource-specific audiences - e.g. Supabase's OAuth server always issues {@code
* aud=authenticated} - so operators list the audience their IdP actually emits here
* (env: {@code MCP_AUTH_ACCEPTEDAUDIENCES}, comma-separated).
*/
private List<String> acceptedAudiences = new ArrayList<>();
/**
* JWT claim whose value is matched against a provisioned Stirling username. Defaults to
* {@code sub}; set to {@code email} or {@code preferred_username} to match how your IdP
* maps users to Stirling accounts.
*/
private String usernameClaim = "sub";
/**
* When {@code true} (default), a validated token is accepted only if its {@link
* #usernameClaim} value resolves to an existing, enabled Stirling user account. Tokens
* whose subject has no Stirling account (or a disabled one) are rejected with HTTP 403.
* Set to {@code false} only if you intentionally want any IdP-valid token to use MCP
* without a local account.
*/
private boolean requireExistingAccount = true;
}
}
/**
* Cluster backplane configuration. All keys live under the top-level {@code cluster.*} prefix
* (e.g. env var {@code CLUSTER_ENABLED}). The master switch is {@link #enabled} and defaults to
@@ -867,6 +1005,10 @@ public class ApplicationProperties {
@Data
public static class Signing {
private boolean enabled = false;
// Signing user-picker scope: 'org' (default) = whole instance, anything else =
// caller's team only (fail-closed). The saas profile pins 'team'.
private String userListScope = "org";
}
}
@@ -1,7 +1,6 @@
package stirling.software.common.model;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.time.LocalDateTime;
import java.time.format.DateTimeFormatter;
import java.util.Locale;
@@ -24,7 +23,7 @@ public class FileInfo {
// Converts the file path string to a Path object.
public Path getFilePathAsPath() {
return Paths.get(filePath);
return Path.of(filePath);
}
// Formats the file size into a human-readable string.
@@ -0,0 +1,191 @@
package stirling.software.common.pdf;
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import stirling.software.jpdfium.text.PageText;
import stirling.software.jpdfium.text.TextChar;
import stirling.software.jpdfium.text.TextLine;
import stirling.software.jpdfium.text.TextWord;
final class HeadingDetector {
private HeadingDetector() {}
/** A heading is at most this many words; longer lines are treated as body text. */
private static final int MAX_HEADING_WORDS = 12;
/**
* Returns the Markdown heading prefix for a line. The decision combines several signals, never
* text matching, so a plain line that merely shares text with a heading is never promoted:
*
* <ul>
* <li><b>Size</b> — dominant glyph font size vs. the document body median (primary signal).
* Some PDFs encode visual size in the text matrix, so every glyph reports ~1.0; for those
* the line height is used as the proxy instead.
* <li><b>Brevity</b> — headings are short labels; a line over {@value #MAX_HEADING_WORDS}
* words is body text regardless of size.
* <li><b>Not a sentence</b> — a line ending in {@code . ! ?} reads as prose, not a heading.
* </ul>
*
* <p>Boldness is deliberately <em>not</em> a heading signal — a bold-but-not-larger line is
* emphasis, not a heading (see {@link #isBoldLabel}); promoting it to {@code #}/{@code ##} is
* the main source of false-positive headings.
*
* <ul>
* <li>size &gt; baseline * 1.4 → {@code "# "}
* <li>size &gt; baseline * 1.2 → {@code "## "}
* <li>otherwise → {@code ""}
* </ul>
*/
static String headingPrefix(TextLine line, float medianBodySize, float medianBodyHeight) {
String text = line.text().strip();
if (text.isEmpty() || wordCount(text) > MAX_HEADING_WORDS || endsLikeSentence(text)) {
return "";
}
float dominant = dominantFontSize(line);
float value;
float baseline;
if (dominant > 2f && medianBodySize > 2f) {
value = dominant;
baseline = medianBodySize;
} else {
value = line.height();
baseline = medianBodyHeight;
}
if (baseline <= 0f) {
return "";
}
float ratio = value / baseline;
if (ratio > 1.4f) {
return "# ";
}
if (ratio > 1.2f) {
return "## ";
}
return "";
}
/**
* True when a line should be emphasised as bold (rendered {@code **like this**}) rather than
* promoted to a heading: it is bold, short, and not a full sentence. Used for bold labels that
* are not large enough to be headings.
*/
static boolean isBoldLabel(TextLine line) {
String text = line.text().strip();
if (text.isEmpty() || wordCount(text) > MAX_HEADING_WORDS || endsLikeSentence(text)) {
return false;
}
return isBold(line);
}
private static int wordCount(String text) {
return text.split("\\s+").length;
}
private static boolean endsLikeSentence(String text) {
char last = text.charAt(text.length() - 1);
return last == '.' || last == '!' || last == '?';
}
/** True when the line's dominant font is bold, inferred from PostScript font names. */
private static boolean isBold(TextLine line) {
Map<String, Integer> counts = new HashMap<>();
for (TextWord word : line.words()) {
for (TextChar ch : word.chars()) {
if (ch.isWhitespace() || ch.isNewline()) {
continue;
}
String name = ch.fontName();
if (name != null && !name.isBlank()) {
counts.merge(name, 1, Integer::sum);
}
}
}
String dominantFont = "";
int max = -1;
for (Map.Entry<String, Integer> e : counts.entrySet()) {
if (e.getValue() > max) {
max = e.getValue();
dominantFont = e.getKey();
}
}
String lower = dominantFont.toLowerCase(java.util.Locale.ROOT);
return lower.contains("bold")
|| lower.contains("black")
|| lower.contains("heavy")
|| lower.contains("semibold");
}
/** Computes the median glyph font size across all pages. */
static float medianFontSize(List<PageText> allPages) {
List<Float> sizes = new ArrayList<>();
for (PageText page : allPages) {
for (TextChar ch : page.chars()) {
if (!ch.isWhitespace() && !ch.isNewline() && ch.fontSize() > 0f) {
sizes.add(ch.fontSize());
}
}
}
return median(sizes, 12f);
}
/** Computes the median TextLine height across all pages. Used when font size is degenerate. */
static float medianLineHeight(List<PageText> allPages) {
List<Float> heights = new ArrayList<>();
for (PageText page : allPages) {
for (TextLine line : page.lines()) {
if (line.height() > 0f && !line.text().isBlank()) {
heights.add(line.height());
}
}
}
return median(heights, 12f);
}
private static float median(List<Float> values, float fallback) {
if (values.isEmpty()) {
return fallback;
}
Collections.sort(values);
int mid = values.size() / 2;
if (values.size() % 2 == 0) {
return (values.get(mid - 1) + values.get(mid)) / 2f;
}
return values.get(mid);
}
/**
* Returns the font size that appears most often (by character count) in the given line. Ties
* are broken in favour of the larger size.
*/
private static float dominantFontSize(TextLine line) {
Map<Float, Integer> counts = new HashMap<>();
for (TextWord word : line.words()) {
for (TextChar ch : word.chars()) {
if (!ch.isWhitespace() && !ch.isNewline() && ch.fontSize() > 0f) {
counts.merge(ch.fontSize(), 1, Integer::sum);
}
}
}
if (counts.isEmpty()) {
return 0f;
}
float dominant = 0f;
int maxCount = -1;
for (Map.Entry<Float, Integer> entry : counts.entrySet()) {
int count = entry.getValue();
float size = entry.getKey();
if (count > maxCount || (count == maxCount && size > dominant)) {
maxCount = count;
dominant = size;
}
}
return dominant;
}
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,82 @@
package stirling.software.common.pdf;
import stirling.software.jpdfium.text.Table;
final class TableRenderer {
private TableRenderer() {}
/** Renders a Table as a GitHub-Flavoured Markdown table string. */
static String render(Table table) {
if (table.rowCount() == 0) {
return "";
}
String[][] grid = table.asGrid();
if (table.rowCount() < 2) {
// No separator row possible — return plain lines
StringBuilder sb = new StringBuilder();
for (int c = 0; c < grid[0].length; c++) {
if (c > 0) sb.append('\n');
sb.append(escape(grid[0][c].trim()));
}
return sb.toString();
}
int cols = grid[0].length;
// Compute column widths: max(3, max content length across all rows)
int[] widths = new int[cols];
for (int c = 0; c < cols; c++) {
widths[c] = 3;
}
for (String[] row : grid) {
for (int c = 0; c < cols; c++) {
String cell = c < row.length ? row[c].trim() : "";
widths[c] = Math.max(widths[c], escape(cell).length());
}
}
StringBuilder sb = new StringBuilder();
// Header row
sb.append(buildRow(grid[0], widths, cols));
sb.append('\n');
// Separator row
sb.append('|');
for (int c = 0; c < cols; c++) {
sb.append('-').append("-".repeat(widths[c])).append('-').append('|');
}
sb.append('\n');
// Data rows
for (int r = 1; r < grid.length; r++) {
sb.append(buildRow(grid[r], widths, cols));
if (r < grid.length - 1) {
sb.append('\n');
}
}
return sb.toString();
}
private static String buildRow(String[] row, int[] widths, int cols) {
StringBuilder sb = new StringBuilder();
sb.append('|');
for (int c = 0; c < cols; c++) {
String cell = c < row.length ? escape(row[c].trim()) : "";
sb.append(' ').append(padRight(cell, widths[c])).append(' ').append('|');
}
return sb.toString();
}
private static String escape(String cell) {
return cell.replace("|", "\\|");
}
private static String padRight(String s, int width) {
if (s.length() >= width) return s;
return s + " ".repeat(width - s.length());
}
}
@@ -5,6 +5,7 @@ import java.io.IOException;
import java.io.InputStream;
import java.io.PipedInputStream;
import java.io.PipedOutputStream;
import java.util.Optional;
import java.util.concurrent.Executors;
import java.util.concurrent.atomic.AtomicReference;
@@ -17,6 +18,7 @@ import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.cluster.FileStore;
import stirling.software.common.util.JobContext;
/**
* Service for storing and retrieving files with unique file IDs. Used by the AutoJobPostMapping
@@ -32,8 +34,10 @@ public class FileStorage {
private final FileOrUploadService fileOrUploadService;
private final FileStore fileStore;
private final Optional<JobOwnershipService> jobOwnershipService;
public String storeFile(MultipartFile file) throws IOException {
String owner = resolveOwner();
// Fast path: when Spring buffered the multipart to disk (typical for large uploads), the
// backing Resource exposes a real File. Hand the Path to the FileStore so it can do a
// file-to-file copy (Linux sendfile, no copy through Java heap) rather than streaming
@@ -48,7 +52,7 @@ public class FileStorage {
if (res != null && res.isFile()) {
try {
FileStore.Stored stored =
fileStore.store(res.getFile().toPath(), file.getOriginalFilename());
fileStore.store(res.getFile().toPath(), file.getOriginalFilename(), owner);
log.debug("Stored file with ID: {} (fast path)", stored.fileId());
return stored.fileId();
} catch (IOException ex) {
@@ -57,40 +61,45 @@ public class FileStorage {
}
}
try (InputStream in = file.getInputStream()) {
FileStore.Stored stored = fileStore.store(in, file.getOriginalFilename());
FileStore.Stored stored = fileStore.store(in, file.getOriginalFilename(), owner);
log.debug("Stored file with ID: {}", stored.fileId());
return stored.fileId();
}
}
public String storeBytes(byte[] bytes, String originalName) throws IOException {
FileStore.Stored stored = fileStore.store(new ByteArrayInputStream(bytes), originalName);
FileStore.Stored stored =
fileStore.store(new ByteArrayInputStream(bytes), originalName, resolveOwner());
log.debug("Stored byte array with ID: {}", stored.fileId());
return stored.fileId();
}
public MultipartFile retrieveFile(String fileId) throws IOException {
enforceOwnership(fileId);
byte[] fileData = fileStore.retrieveBytes(fileId);
return fileOrUploadService.toMockMultipartFile(fileId, fileData);
}
public byte[] retrieveBytes(String fileId) throws IOException {
enforceOwnership(fileId);
return fileStore.retrieveBytes(fileId);
}
public InputStream retrieveInputStream(String fileId) throws IOException {
enforceOwnership(fileId);
return fileStore.retrieve(fileId);
}
public StoredFile storeInputStream(InputStream inputStream, String originalName)
throws IOException {
FileStore.Stored stored = fileStore.store(inputStream, originalName);
FileStore.Stored stored = fileStore.store(inputStream, originalName, resolveOwner());
log.debug("Stored input stream with ID: {}", stored.fileId());
return new StoredFile(stored.fileId(), stored.size());
}
public String storeFromStreamingBody(StreamingResponseBody body, String originalName)
throws IOException {
String owner = resolveOwner();
// Hold Throwable not IOException: an unchecked failure (NPE, IllegalState, OOM, etc.)
// from the body writer would otherwise close the pipe with EOF and the consumer would
// return a truncated file with no error surfaced to the caller.
@@ -115,7 +124,7 @@ public class FileStorage {
}
}
});
FileStore.Stored stored = fileStore.store(in, originalName);
FileStore.Stored stored = fileStore.store(in, originalName, owner);
Throwable writerErr = bodyError.get();
if (writerErr != null) {
// Body failed mid-write: the FileStore persisted a truncated entry.
@@ -159,21 +168,62 @@ public class FileStorage {
public String storeFromResource(Resource resource, String originalName) throws IOException {
try (InputStream in = resource.getInputStream()) {
FileStore.Stored stored = fileStore.store(in, originalName);
FileStore.Stored stored = fileStore.store(in, originalName, resolveOwner());
log.debug("Stored Resource with ID: {}", stored.fileId());
return stored.fileId();
}
}
public boolean deleteFile(String fileId) {
enforceOwnership(fileId);
return fileStore.delete(fileId);
}
public boolean fileExists(String fileId) {
enforceOwnership(fileId);
return fileStore.exists(fileId);
}
public long getFileSize(String fileId) throws IOException {
enforceOwnership(fileId);
return fileStore.size(fileId);
}
private String resolveOwner() {
String propagated = JobContext.getOwner();
if (propagated != null) {
return propagated;
}
return jobOwnershipService.flatMap(JobOwnershipService::getCurrentUserId).orElse(null);
}
private void enforceOwnership(String fileId) {
if (jobOwnershipService.isEmpty()) {
return;
}
Optional<String> currentUser = jobOwnershipService.get().getCurrentUserId();
if (currentUser.isEmpty()) {
return;
}
String owner;
try {
owner = fileStore.getOwner(fileId);
} catch (IOException e) {
log.warn("Failed to read owner for file {}: {}", fileId, e.getMessage());
throw new SecurityException(
"Access denied: could not verify ownership of the requested file");
}
if (owner == null) {
return;
}
if (!owner.equals(currentUser.get())) {
log.warn(
"Access denied: user {} attempted to access file {} owned by {}",
currentUser.get(),
fileId,
owner);
throw new SecurityException(
"Access denied: you do not have permission to access this file");
}
}
}
@@ -50,6 +50,16 @@ public class InternalApiClient {
"^/api/v1/(general|misc|security|convert|filter)(/[A-Za-z0-9_-]+)+$"
+ "|^/api/v1/ai/tools(/[A-Za-z0-9_-]+)+$");
/**
* Marker propagated on every internal sub-step dispatch so the saas PAYG interceptor classifies
* the call as {@code BillingCategory.AUTOMATION}. By construction every {@link
* InternalApiClient#post} caller is an automation surface (pipeline executor, AI workflow,
* policy runner) running a child tool inside a parent automation flow — see the saas {@code
* PaygChargeInterceptor.determineCategory} precedence chain, where this header dominates any
* per-tool {@code @RequiresFeature} annotation.
*/
public static final String AUTOMATION_HEADER = "X-Stirling-Automation";
private final ServletContext servletContext;
private final UserServiceInterface userService;
private final TempFileManager tempFileManager;
@@ -96,7 +106,23 @@ public class InternalApiClient {
if (apiKey != null && !apiKey.isEmpty()) {
headers.add("X-API-KEY", apiKey);
}
// Tag the sub-step as automation so PAYG bills it under AUTOMATION regardless of which
// tool-level @RequiresFeature annotation the dispatched controller carries (e.g. an AI-OCR
// step inside a policy run must bill as AUTOMATION, not AI). Set unconditionally because
// every caller of this dispatcher is an automation surface by design.
headers.add(AUTOMATION_HEADER, "true");
// A no-file ai/tools call (e.g. create-pdf-from-html-agent) sends only string params, so
// without this RestTemplate would use urlencoded instead of the multipart the controller
// expects. File-bearing calls get the right multipart content-type from RestTemplate.
boolean isAiTool = endpointPath.startsWith("/api/v1/ai/tools/");
boolean hasFilePart =
body.values().stream()
.flatMap(java.util.List::stream)
.anyMatch(v -> v instanceof Resource);
if (isAiTool && !hasFilePart) {
headers.setContentType(MediaType.MULTIPART_FORM_DATA);
}
HttpEntity<MultiValueMap<String, Object>> entity = new HttpEntity<>(body, headers);
RequestCallback requestCallback = restTemplate.httpEntityCallback(entity, Resource.class);
@@ -89,6 +89,11 @@ public class JobExecutorService {
String jobId = scopedJobKey;
final String jobOwner =
jobOwnershipService != null
? jobOwnershipService.getCurrentUserId().orElse(null)
: null;
long timeoutToUse = customTimeoutMs > 0 ? customTimeoutMs : effectiveTimeoutMs;
log.debug(
@@ -119,6 +124,7 @@ public class JobExecutorService {
try {
stirling.software.common.util.JobContext.setJobId(
capturedJobIdForQueue);
stirling.software.common.util.JobContext.setOwner(jobOwner);
Object result = work.get();
processJobResult(capturedJobIdForQueue, result);
return result;
@@ -153,6 +159,7 @@ public class JobExecutorService {
timeoutToUse);
stirling.software.common.util.JobContext.setJobId(capturedJobId);
stirling.software.common.util.JobContext.setOwner(jobOwner);
Object result = executeWithTimeout(() -> work.get(), timeoutToUse);
processJobResult(capturedJobId, result);
} catch (TimeoutException te) {
@@ -3,7 +3,6 @@ package stirling.software.common.service;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
@@ -35,7 +34,7 @@ public class MobileScannerService {
public MobileScannerService() throws IOException {
// Create temp directory for mobile scanner uploads
this.tempDirectory =
Paths.get(System.getProperty("java.io.tmpdir"), "stirling-mobile-scanner");
Path.of(System.getProperty("java.io.tmpdir"), "stirling-mobile-scanner");
Files.createDirectories(tempDirectory);
log.info("Mobile scanner temp directory: {}", tempDirectory);
}
@@ -8,7 +8,7 @@ import java.lang.management.OperatingSystemMXBean;
import java.lang.management.RuntimeMXBean;
import java.lang.management.ThreadMXBean;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.nio.file.Path;
import java.util.HashMap;
import java.util.Locale;
import java.util.Map;
@@ -160,7 +160,7 @@ public class PostHogService {
}
private boolean isRunningInDocker() {
return Files.exists(Paths.get("/.dockerenv"));
return Files.exists(Path.of("/.dockerenv"));
}
private Map<String, Object> getDockerMetrics() {
@@ -1,11 +1,21 @@
package stirling.software.common.service;
import java.util.List;
/** Provides metadata about tool endpoints for internal dispatch. */
public interface ToolMetadataService {
/** Returns true if the given operation path accepts multiple input files. */
boolean isMultiInput(String operationPath);
/**
* Returns the file extensions (lowercase, no leading dot, e.g. {@code "pdf"}) that the
* operation accepts as input ({@code output=false}) or produces as output ({@code
* output=true}), derived from the endpoint's declared type. Returns {@code null} when the
* endpoint declares no specific type, which callers should treat as "any type accepted".
*/
List<String> getExtensionTypes(boolean output, String operationPath);
/**
* Returns true when the endpoint's ZIP response is a transport for multiple typed results and
* should be unpacked: multi-output endpoints (Type:SIMO / Type:MIMO) and wrapper declarations
@@ -255,7 +255,7 @@ public class GeneralUtils {
String pattern = locationPattern;
if (pattern.startsWith("file:")) {
String rawPath = pattern.substring(5).replace("\\*", "").replace("/*", "");
Path normalizePath = Paths.get(rawPath).normalize();
Path normalizePath = Path.of(rawPath).normalize();
pattern = "file:" + normalizePath.toString().replace("\\", "/") + "/*";
}
return ResourcePatternUtils.getResourcePatternResolver(resourceLoader)
@@ -837,7 +837,7 @@ public class GeneralUtils {
}
public boolean createDir(String path) {
Path folder = Paths.get(path);
Path folder = Path.of(path);
if (!Files.exists(folder)) {
try {
Files.createDirectories(folder);
@@ -867,7 +867,7 @@ public class GeneralUtils {
public void saveKeyToSettings(String key, Object newValue) throws IOException {
String[] keyArray = key.split("\\.");
Path settingsPath = Paths.get(InstallationPathConfig.getSettingsPath());
Path settingsPath = Path.of(InstallationPathConfig.getSettingsPath());
YamlHelper settingsYaml = new YamlHelper(settingsPath);
settingsYaml.updateValue(Arrays.asList(keyArray), newValue);
settingsYaml.saveOverride(settingsPath);
@@ -888,7 +888,7 @@ public class GeneralUtils {
return;
}
Path settingsPath = Paths.get(InstallationPathConfig.getSettingsPath());
Path settingsPath = Path.of(InstallationPathConfig.getSettingsPath());
YamlHelper settingsYaml = new YamlHelper(settingsPath);
// Apply all updates to the same YamlHelper instance
@@ -974,11 +974,11 @@ public class GeneralUtils {
*/
public void extractPipeline() throws IOException {
Path pipelineDir =
Paths.get(InstallationPathConfig.getPipelinePath(), DEFAULT_WEBUI_CONFIGS_DIR);
Path.of(InstallationPathConfig.getPipelinePath(), DEFAULT_WEBUI_CONFIGS_DIR);
Files.createDirectories(pipelineDir);
for (String name : DEFAULT_VALID_PIPELINE) {
if (!Paths.get(name).getFileName().toString().equals(name)) {
if (!Path.of(name).getFileName().toString().equals(name)) {
log.error("Invalid pipeline file name: {}", name);
throw new IllegalArgumentException("Invalid pipeline file name: " + name);
}
@@ -1014,7 +1014,7 @@ public class GeneralUtils {
throw new IllegalArgumentException(
"scriptName must not contain path traversal characters");
}
if (!Paths.get(scriptName).getFileName().toString().equals(scriptName)) {
if (!Path.of(scriptName).getFileName().toString().equals(scriptName)) {
throw new IllegalArgumentException(
"scriptName must not contain path traversal characters");
}
@@ -1024,7 +1024,7 @@ public class GeneralUtils {
"scriptName must be either 'png_to_webp.py' or 'split_photos.py'");
}
Path scriptsDir = Paths.get(InstallationPathConfig.getScriptsPath(), PYTHON_SCRIPTS_DIR);
Path scriptsDir = Path.of(InstallationPathConfig.getScriptsPath(), PYTHON_SCRIPTS_DIR);
Files.createDirectories(scriptsDir);
Path target = scriptsDir.resolve(scriptName);
@@ -4,7 +4,6 @@ import java.io.File;
import java.net.URISyntaxException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import lombok.extern.slf4j.Slf4j;
@@ -20,7 +19,7 @@ public class JarPathUtil {
public static Path currentJar() {
try {
Path jar =
Paths.get(
Path.of(
JarPathUtil.class
.getProtectionDomain()
.getCodeSource()
@@ -61,14 +60,14 @@ public class JarPathUtil {
}
// Location 2: ./build/libs/ (development build)
possibleLocations[1] = Paths.get("build", "libs", "restart-helper.jar").toAbsolutePath();
possibleLocations[1] = Path.of("build", "libs", "restart-helper.jar").toAbsolutePath();
// Location 3: app/common/build/libs/ (multi-module build)
possibleLocations[2] =
Paths.get("app", "common", "build", "libs", "restart-helper.jar").toAbsolutePath();
Path.of("app", "common", "build", "libs", "restart-helper.jar").toAbsolutePath();
// Location 4: Current working directory
possibleLocations[3] = Paths.get("restart-helper.jar").toAbsolutePath();
possibleLocations[3] = Path.of("restart-helper.jar").toAbsolutePath();
// Check each location
for (Path location : possibleLocations) {
@@ -1,8 +1,9 @@
package stirling.software.common.util;
/** Thread-local context for passing job ID across async boundaries */
/** Thread-local context for passing job ID and owner across async boundaries */
public class JobContext {
private static final ThreadLocal<String> CURRENT_JOB_ID = new ThreadLocal<>();
private static final ThreadLocal<String> CURRENT_OWNER = new ThreadLocal<>();
public static void setJobId(String jobId) {
CURRENT_JOB_ID.set(jobId);
@@ -12,7 +13,16 @@ public class JobContext {
return CURRENT_JOB_ID.get();
}
public static void setOwner(String owner) {
CURRENT_OWNER.set(owner);
}
public static String getOwner() {
return CURRENT_OWNER.get();
}
public static void clear() {
CURRENT_JOB_ID.remove();
CURRENT_OWNER.remove();
}
}
@@ -0,0 +1,310 @@
package stirling.software.common.util;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.Locale;
import java.util.Set;
import java.util.zip.ZipEntry;
import java.util.zip.ZipInputStream;
import java.util.zip.ZipOutputStream;
import javax.xml.XMLConstants;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.parsers.ParserConfigurationException;
import javax.xml.transform.OutputKeys;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerException;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamResult;
import org.springframework.stereotype.Component;
import org.w3c.dom.Document;
import org.w3c.dom.Element;
import org.w3c.dom.NamedNodeMap;
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;
import org.xml.sax.SAXException;
import io.github.pixee.security.ZipSecurity;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.service.SsrfProtectionService;
// Strips external refs from OOXML/ODF uploads so LibreOffice can't be made to fetch them.
@Component
@Slf4j
public class OfficeDocumentSanitizer {
private static final Set<String> OOXML_EXTENSIONS =
Set.of(
"docx", "docm", "dotx", "dotm", "xlsx", "xlsm", "xltx", "xltm", "pptx", "pptm",
"potx", "potm", "ppsx", "ppsm");
private static final Set<String> ODF_EXTENSIONS =
Set.of(
"odt", "ott", "ods", "ots", "odp", "otp", "odg", "otg", "odf", "odc", "odi",
"odm");
private static final Set<String> ODF_XML_PARTS =
Set.of("content.xml", "styles.xml", "meta.xml", "settings.xml");
private final SsrfProtectionService ssrfProtectionService;
private final ApplicationProperties applicationProperties;
public OfficeDocumentSanitizer(
SsrfProtectionService ssrfProtectionService,
ApplicationProperties applicationProperties) {
this.ssrfProtectionService = ssrfProtectionService;
this.applicationProperties = applicationProperties;
}
public boolean isSanitizableExtension(String extension) {
if (extension == null) {
return false;
}
String lower = extension.toLowerCase(Locale.ROOT);
return OOXML_EXTENSIONS.contains(lower) || ODF_EXTENSIONS.contains(lower);
}
public byte[] sanitize(byte[] documentBytes, String extension) throws IOException {
if (documentBytes == null || documentBytes.length == 0) {
throw new IOException("Office document input is empty or null");
}
if (applicationProperties.getSystem().isDisableSanitize()) {
log.debug("Office document sanitization disabled by configuration");
return documentBytes;
}
if (!isSanitizableExtension(extension)) {
return documentBytes;
}
ByteArrayOutputStream out = new ByteArrayOutputStream(documentBytes.length);
try (ZipInputStream zipIn =
ZipSecurity.createHardenedInputStream(
new ByteArrayInputStream(documentBytes));
ZipOutputStream zipOut = new ZipOutputStream(out)) {
ZipEntry entry;
while ((entry = zipIn.getNextEntry()) != null) {
String name = entry.getName();
byte[] bytes = entry.isDirectory() ? new byte[0] : zipIn.readAllBytes();
if (!entry.isDirectory()) {
bytes = sanitizeEntry(name, bytes);
}
ZipEntry outEntry = new ZipEntry(name);
if (entry.getComment() != null) {
outEntry.setComment(entry.getComment());
}
if (entry.getExtra() != null) {
outEntry.setExtra(entry.getExtra());
}
zipOut.putNextEntry(outEntry);
if (!entry.isDirectory()) {
zipOut.write(bytes);
}
zipOut.closeEntry();
}
}
return out.toByteArray();
}
private byte[] sanitizeEntry(String entryName, byte[] entryBytes) {
String lower = entryName.toLowerCase(Locale.ROOT);
try {
if (lower.endsWith(".rels")) {
return sanitizeOoxmlRels(entryBytes);
}
if (isOdfXmlPart(lower)) {
return sanitizeOdfXml(entryBytes);
}
} catch (ParserConfigurationException
| SAXException
| IOException
| TransformerException e) {
log.warn(
"Failed to parse XML part '{}' for sanitization, leaving as-is: {}",
entryName,
e.getMessage());
}
return entryBytes;
}
private boolean isOdfXmlPart(String lowerName) {
int slash = lowerName.lastIndexOf('/');
String base = slash >= 0 ? lowerName.substring(slash + 1) : lowerName;
return ODF_XML_PARTS.contains(base);
}
private byte[] sanitizeOoxmlRels(byte[] xmlBytes)
throws IOException, ParserConfigurationException, SAXException, TransformerException {
Document doc = parseSecurely(xmlBytes);
Element root = doc.getDocumentElement();
if (root == null) {
return xmlBytes;
}
NodeList relationships = root.getElementsByTagNameNS("*", "Relationship");
List<Node> toRemove = new ArrayList<>();
for (int i = 0; i < relationships.getLength(); i++) {
Node node = relationships.item(i);
NamedNodeMap attrs = node.getAttributes();
if (attrs == null) {
continue;
}
Node targetMode = attrs.getNamedItem("TargetMode");
if (targetMode == null || !"external".equalsIgnoreCase(targetMode.getNodeValue())) {
continue;
}
Node target = attrs.getNamedItem("Target");
String targetValue = target == null ? "" : target.getNodeValue();
if (isAdminAllowed(targetValue)) {
continue;
}
log.warn(
"Stripping OOXML external relationship target: {}",
truncateForLog(targetValue));
toRemove.add(node);
}
if (toRemove.isEmpty()) {
return xmlBytes;
}
for (Node n : toRemove) {
n.getParentNode().removeChild(n);
}
return serializeDocument(doc);
}
private byte[] sanitizeOdfXml(byte[] xmlBytes)
throws IOException, ParserConfigurationException, SAXException, TransformerException {
Document doc = parseSecurely(xmlBytes);
Element root = doc.getDocumentElement();
if (root == null) {
return xmlBytes;
}
boolean modified = stripExternalHrefs(root);
if (!modified) {
return xmlBytes;
}
return serializeDocument(doc);
}
private boolean stripExternalHrefs(Node node) {
boolean modified = false;
if (node.getNodeType() == Node.ELEMENT_NODE) {
NamedNodeMap attrs = node.getAttributes();
List<String> hrefAttrsToRemove = new ArrayList<>();
for (int i = 0; i < attrs.getLength(); i++) {
Node attr = attrs.item(i);
String name = attr.getNodeName();
if (name == null) {
continue;
}
String lower = name.toLowerCase(Locale.ROOT);
if (!(lower.equals("xlink:href")
|| lower.endsWith(":href")
|| lower.equals("href"))) {
continue;
}
String value = attr.getNodeValue();
if (!isExternalUrl(value)) {
continue;
}
if (isAdminAllowed(value)) {
continue;
}
log.warn(
"Stripping ODF external href attribute ({}): {}",
name,
truncateForLog(value));
hrefAttrsToRemove.add(name);
}
Element element = (Element) node;
for (String attrName : hrefAttrsToRemove) {
element.removeAttribute(attrName);
modified = true;
}
}
NodeList children = node.getChildNodes();
for (int i = 0; i < children.getLength(); i++) {
if (stripExternalHrefs(children.item(i))) {
modified = true;
}
}
return modified;
}
private boolean isExternalUrl(String url) {
if (url == null) {
return false;
}
String trimmed = url.trim().toLowerCase(Locale.ROOT);
if (trimmed.isEmpty() || trimmed.startsWith("#") || trimmed.startsWith("../")) {
return false;
}
return trimmed.startsWith("http://")
|| trimmed.startsWith("https://")
|| trimmed.startsWith("ftp://")
|| trimmed.startsWith("ftps://")
|| trimmed.startsWith("file:")
|| trimmed.startsWith("smb:")
|| trimmed.startsWith("\\\\")
|| trimmed.startsWith("//");
}
// Preserved only with an explicit allowedDomains entry; MEDIUM default would admit public URLs.
private boolean isAdminAllowed(String url) {
if (ssrfProtectionService == null || url == null || url.isBlank()) {
return false;
}
ApplicationProperties.Html.UrlSecurity config =
applicationProperties.getSystem().getHtml().getUrlSecurity();
if (config == null
|| config.getAllowedDomains() == null
|| config.getAllowedDomains().isEmpty()) {
return false;
}
return ssrfProtectionService.isUrlAllowed(url);
}
private Document parseSecurely(byte[] xmlBytes)
throws ParserConfigurationException, SAXException, IOException {
DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
factory.setFeature(XMLConstants.FEATURE_SECURE_PROCESSING, true);
factory.setFeature("http://apache.org/xml/features/disallow-doctype-decl", true);
factory.setFeature("http://xml.org/sax/features/external-general-entities", false);
factory.setFeature("http://xml.org/sax/features/external-parameter-entities", false);
factory.setFeature("http://apache.org/xml/features/nonvalidating/load-external-dtd", false);
factory.setXIncludeAware(false);
factory.setExpandEntityReferences(false);
factory.setNamespaceAware(true);
DocumentBuilder builder = factory.newDocumentBuilder();
return builder.parse(new ByteArrayInputStream(xmlBytes));
}
private byte[] serializeDocument(Document doc) throws TransformerException {
TransformerFactory tf = TransformerFactory.newInstance();
tf.setFeature(XMLConstants.FEATURE_SECURE_PROCESSING, true);
Transformer transformer = tf.newTransformer();
transformer.setOutputProperty(OutputKeys.ENCODING, "UTF-8");
transformer.setOutputProperty(OutputKeys.INDENT, "no");
transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "no");
ByteArrayOutputStream baos = new ByteArrayOutputStream();
transformer.transform(new DOMSource(doc), new StreamResult(baos));
return baos.toByteArray();
}
private String truncateForLog(String value) {
if (value == null) {
return "null";
}
return value.length() > 80 ? value.substring(0, 80) + "..." : value;
}
}
@@ -0,0 +1,481 @@
package stirling.software.SPDF.config;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
import java.util.List;
import java.util.Set;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test;
import stirling.software.SPDF.config.EndpointConfiguration.DisableReason;
import stirling.software.SPDF.config.EndpointConfiguration.EndpointAvailability;
import stirling.software.common.model.ApplicationProperties;
/**
* Unit tests for {@link EndpointConfiguration}. The class wires up its endpoint/group registry in
* {@code init()} during construction and then applies environment overrides. We build it with a
* real {@link ApplicationProperties} (whose System/Endpoints sub-objects are non-null by default)
* so the constructor runs cleanly without any mocking.
*/
class EndpointConfigurationGapTest {
private ApplicationProperties applicationProperties;
/**
* Construct an EndpointConfiguration with the given pro flag and current applicationProperties.
*/
private EndpointConfiguration build(boolean runningProOrHigher) {
return new EndpointConfiguration(applicationProperties, runningProOrHigher);
}
/** Default config: not pro, no removals, url-to-pdf disabled (default System flag is false). */
private EndpointConfiguration buildDefault() {
return build(false);
}
@BeforeEach
void setUp() {
applicationProperties = new ApplicationProperties();
}
@Nested
@DisplayName("endpointKeyForUri (static)")
class EndpointKeyForUriTests {
@Test
@DisplayName("returns null for null uri")
void nullUri() {
assertNull(EndpointConfiguration.endpointKeyForUri(null));
}
@Test
@DisplayName("returns null when uri does not contain /api/v1")
void notApiPath() {
assertNull(EndpointConfiguration.endpointKeyForUri("/foo/bar/baz"));
assertNull(EndpointConfiguration.endpointKeyForUri("https://example.com/home"));
}
@Test
@DisplayName("returns null when uri has too few path segments")
void tooFewSegments() {
// "/api/v1/general" splits to ["", "api", "v1", "general"] -> length 4, not > 4
assertNull(EndpointConfiguration.endpointKeyForUri("/api/v1/general"));
}
@Test
@DisplayName("extracts plain endpoint key from a standard /api/v1/<group>/<endpoint> uri")
void plainEndpoint() {
assertEquals(
"remove-pages",
EndpointConfiguration.endpointKeyForUri("/api/v1/general/remove-pages"));
}
@Test
@DisplayName("builds a <from>-to-<to> key for convert endpoints")
void convertEndpoint() {
assertEquals(
"pdf-to-img",
EndpointConfiguration.endpointKeyForUri("/api/v1/convert/pdf/img"));
}
@Test
@DisplayName("convert path without a target segment falls back to the segment after group")
void convertWithoutTarget() {
// "/api/v1/convert/pdf" -> length 5, the convert branch needs length > 5
assertEquals("pdf", EndpointConfiguration.endpointKeyForUri("/api/v1/convert/pdf"));
}
}
@Nested
@DisplayName("enable / disable endpoint")
class EnableDisableEndpointTests {
@Test
@DisplayName("a freshly registered endpoint is enabled by default")
void enabledByDefault() {
EndpointConfiguration config = buildDefault();
assertTrue(config.isEndpointEnabled("merge-pdfs"));
}
@Test
@DisplayName("disableEndpoint marks the endpoint disabled")
void disableEndpoint() {
EndpointConfiguration config = buildDefault();
config.disableEndpoint("merge-pdfs");
assertFalse(config.isEndpointEnabled("merge-pdfs"));
}
@Test
@DisplayName("enableEndpoint re-enables a previously disabled endpoint")
void reEnableEndpoint() {
EndpointConfiguration config = buildDefault();
config.disableEndpoint("merge-pdfs");
assertFalse(config.isEndpointEnabled("merge-pdfs"));
config.enableEndpoint("merge-pdfs");
assertTrue(config.isEndpointEnabled("merge-pdfs"));
}
@Test
@DisplayName("leading slash is normalized away on disable")
void leadingSlashNormalizedOnDisable() {
EndpointConfiguration config = buildDefault();
config.disableEndpoint("/merge-pdfs");
// both forms resolve to the same key
assertFalse(config.isEndpointEnabled("merge-pdfs"));
assertFalse(config.isEndpointEnabled("/merge-pdfs"));
}
@Test
@DisplayName("isEndpointEnabled tolerates a leading slash on the query")
void leadingSlashOnQuery() {
EndpointConfiguration config = buildDefault();
assertTrue(config.isEndpointEnabled("/merge-pdfs"));
}
@Test
@DisplayName("disabling clears with enable, removing the disable reason")
void enableClearsReason() {
EndpointConfiguration config = buildDefault();
config.disableEndpoint("split-pages", DisableReason.DEPENDENCY);
assertEquals(
DisableReason.DEPENDENCY,
config.getEndpointAvailability("split-pages").getReason());
config.enableEndpoint("split-pages");
EndpointAvailability availability = config.getEndpointAvailability("split-pages");
assertTrue(availability.isEnabled());
assertNull(availability.getReason());
}
}
@Nested
@DisplayName("isEndpointEnabledForUri")
class IsEndpointEnabledForUriTests {
@Test
@DisplayName("translates a /api/v1 uri to a key and reports its status")
void translatesUri() {
EndpointConfiguration config = buildDefault();
assertTrue(config.isEndpointEnabledForUri("/api/v1/general/merge-pdfs"));
config.disableEndpoint("merge-pdfs");
assertFalse(config.isEndpointEnabledForUri("/api/v1/general/merge-pdfs"));
}
@Test
@DisplayName("falls back to treating a non-api uri as a raw key")
void fallsBackToRawKey() {
EndpointConfiguration config = buildDefault();
config.disableEndpoint("merge-pdfs");
// non-api path: key resolution returns null, so the uri itself is used as the key
assertFalse(config.isEndpointEnabledForUri("merge-pdfs"));
}
}
@Nested
@DisplayName("group enable / disable")
class GroupTests {
@Test
@DisplayName("a functional group with all endpoints enabled reports enabled")
void functionalGroupEnabled() {
EndpointConfiguration config = buildDefault();
assertTrue(config.isGroupEnabled("PageOps"));
}
@Test
@DisplayName("disabling a functional group cascades to all its endpoints")
void disableFunctionalGroupCascades() {
EndpointConfiguration config = buildDefault();
config.disableGroup("PageOps");
assertFalse(config.isGroupEnabled("PageOps"));
assertFalse(config.isEndpointEnabled("remove-pages"));
assertFalse(config.isEndpointEnabled("split-pages"));
}
@Test
@DisplayName("re-enabling a functional group re-enables its endpoints")
void enableFunctionalGroupRestores() {
EndpointConfiguration config = buildDefault();
config.disableGroup("PageOps");
assertFalse(config.isEndpointEnabled("remove-pages"));
config.enableGroup("PageOps");
assertTrue(config.isEndpointEnabled("remove-pages"));
assertTrue(config.isGroupEnabled("PageOps"));
}
@Test
@DisplayName("a functional group with one disabled endpoint is not enabled")
void functionalGroupWithDisabledEndpoint() {
EndpointConfiguration config = buildDefault();
config.disableEndpoint("remove-pages");
assertFalse(config.isGroupEnabled("PageOps"));
}
@Test
@DisplayName("disabledGroups reflects disabled groups and getDisabledGroups returns a copy")
void getDisabledGroupsReturnsCopy() {
EndpointConfiguration config = buildDefault();
config.disableGroup("PageOps");
Set<String> disabled = config.getDisabledGroups();
assertTrue(disabled.contains("PageOps"));
// mutating the returned set must not affect internal state
disabled.clear();
assertTrue(config.getDisabledGroups().contains("PageOps"));
}
@Test
@DisplayName("an unknown group with no endpoints is not enabled")
void unknownGroupNotEnabled() {
EndpointConfiguration config = buildDefault();
assertFalse(config.isGroupEnabled("NoSuchGroupXyz"));
}
}
@Nested
@DisplayName("tool group semantics")
class ToolGroupTests {
@Test
@DisplayName("a tool group is enabled until explicitly disabled")
void toolGroupEnabledUntilDisabled() {
EndpointConfiguration config = buildDefault();
assertTrue(config.isGroupEnabled("qpdf"));
config.disableGroup("qpdf");
assertFalse(config.isGroupEnabled("qpdf"));
}
@Test
@DisplayName("disabling a tool group does NOT cascade to its endpoints directly")
void toolGroupNoCascade() {
EndpointConfiguration config = buildDefault();
// repair has alternatives (qpdf, Ghostscript); disabling only qpdf keeps it enabled
config.disableGroup("qpdf");
assertTrue(config.isEndpointEnabled("repair"));
}
@Test
@DisplayName("endpoint with alternatives is disabled only when all tool groups are gone")
void allAlternativesDisabled() {
EndpointConfiguration config = buildDefault();
config.disableGroup("qpdf");
config.disableGroup("Ghostscript");
// repair's only alternatives are qpdf and Ghostscript
assertFalse(config.isEndpointEnabled("repair"));
}
@Test
@DisplayName("endpoint with a still-enabled alternative stays enabled")
void oneAlternativeRemains() {
EndpointConfiguration config = buildDefault();
// compress-pdf alternatives: qpdf, Ghostscript, Java
config.disableGroup("qpdf");
config.disableGroup("Ghostscript");
assertTrue(config.isEndpointEnabled("compress-pdf"));
config.disableGroup("Java");
assertFalse(config.isEndpointEnabled("compress-pdf"));
}
@Test
@DisplayName("single-dependency endpoint (no alternatives) disabled when its tool group is")
void singleDependencyDisabled() {
EndpointConfiguration config = buildDefault();
// pdf-to-epub depends on Calibre, no alternatives registered
assertTrue(config.isEndpointEnabled("pdf-to-epub"));
config.disableGroup("Calibre");
assertFalse(config.isEndpointEnabled("pdf-to-epub"));
}
}
@Nested
@DisplayName("addEndpointToGroup / addEndpointAlternative")
class RegistrationTests {
@Test
@DisplayName("addEndpointToGroup makes the endpoint part of the group")
void addEndpointToGroup() {
EndpointConfiguration config = buildDefault();
config.addEndpointToGroup("CustomGroup", "custom-endpoint");
Set<String> endpoints = config.getEndpointsForGroup("CustomGroup");
assertTrue(endpoints.contains("custom-endpoint"));
}
@Test
@DisplayName("disabling a custom functional group disables its added endpoint")
void customFunctionalGroupCascades() {
EndpointConfiguration config = buildDefault();
config.addEndpointToGroup("CustomGroup", "custom-endpoint");
assertTrue(config.isEndpointEnabled("custom-endpoint"));
config.disableGroup("CustomGroup");
assertFalse(config.isEndpointEnabled("custom-endpoint"));
}
@Test
@DisplayName("getEndpointsForGroup returns an empty set for unknown groups")
void unknownGroupEmptySet() {
EndpointConfiguration config = buildDefault();
Set<String> endpoints = config.getEndpointsForGroup("NoSuchGroupXyz");
assertNotNull(endpoints);
assertTrue(endpoints.isEmpty());
}
}
@Nested
@DisplayName("getEndpointAvailability / determineDisableReason")
class AvailabilityTests {
@Test
@DisplayName("an enabled endpoint has a null disable reason")
void enabledHasNullReason() {
EndpointConfiguration config = buildDefault();
EndpointAvailability availability = config.getEndpointAvailability("merge-pdfs");
assertTrue(availability.isEnabled());
assertNull(availability.getReason());
}
@Test
@DisplayName("explicit disable preserves the supplied reason")
void explicitDisableReason() {
EndpointConfiguration config = buildDefault();
config.disableEndpoint("merge-pdfs", DisableReason.DEPENDENCY);
EndpointAvailability availability = config.getEndpointAvailability("merge-pdfs");
assertFalse(availability.isEnabled());
assertEquals(DisableReason.DEPENDENCY, availability.getReason());
}
@Test
@DisplayName("default disableEndpoint reason is CONFIG")
void defaultDisableReasonIsConfig() {
EndpointConfiguration config = buildDefault();
config.disableEndpoint("merge-pdfs");
assertEquals(
DisableReason.CONFIG, config.getEndpointAvailability("merge-pdfs").getReason());
}
@Test
@DisplayName("endpoint disabled via functional group reports the group's reason")
void functionalGroupReason() {
EndpointConfiguration config = buildDefault();
config.disableGroup("PageOps", DisableReason.DEPENDENCY);
EndpointAvailability availability = config.getEndpointAvailability("crop");
assertFalse(availability.isEnabled());
// crop is disabled both via group cascade and group membership; reason is DEPENDENCY
assertEquals(DisableReason.DEPENDENCY, availability.getReason());
}
}
@Nested
@DisplayName("getAllEndpoints")
class GetAllEndpointsTests {
@Test
@DisplayName("aggregates endpoints across all groups")
void aggregatesAcrossGroups() {
EndpointConfiguration config = buildDefault();
Set<String> all = config.getAllEndpoints();
assertTrue(all.contains("merge-pdfs"));
assertTrue(all.contains("compress-pdf"));
assertTrue(all.contains("ocr-pdf"));
assertFalse(all.isEmpty());
}
@Test
@DisplayName("custom endpoints registered after init appear in getAllEndpoints")
void includesCustomEndpoints() {
EndpointConfiguration config = buildDefault();
config.addEndpointToGroup("CustomGroup", "brand-new-endpoint");
assertTrue(config.getAllEndpoints().contains("brand-new-endpoint"));
}
}
@Nested
@DisplayName("environment / constructor driven configuration")
class EnvironmentConfigTests {
@Test
@DisplayName("url-to-pdf is disabled when enableUrlToPDF is false (default)")
void urlToPdfDisabledByDefault() {
EndpointConfiguration config = buildDefault();
assertFalse(config.isEndpointEnabled("url-to-pdf"));
}
@Test
@DisplayName("url-to-pdf stays enabled when enableUrlToPDF is true")
void urlToPdfEnabledWhenFlagSet() {
applicationProperties.getSystem().setEnableUrlToPDF(true);
EndpointConfiguration config = build(false);
assertTrue(config.isEndpointEnabled("url-to-pdf"));
}
@Test
@DisplayName("endpoints.toRemove disables the listed endpoints at construction")
void endpointsToRemove() {
applicationProperties
.getEndpoints()
.setToRemove(List.of(" merge-pdfs ", "split-pages"));
EndpointConfiguration config = build(false);
// values are trimmed before disabling
assertFalse(config.isEndpointEnabled("merge-pdfs"));
assertFalse(config.isEndpointEnabled("split-pages"));
}
@Test
@DisplayName("endpoints.groupsToRemove disables the listed groups at construction")
void groupsToRemove() {
applicationProperties.getEndpoints().setGroupsToRemove(List.of(" PageOps "));
EndpointConfiguration config = build(false);
assertTrue(config.getDisabledGroups().contains("PageOps"));
assertFalse(config.isEndpointEnabled("remove-pages"));
}
@Test
@DisplayName("non-pro build disables the enterprise group")
void nonProDisablesEnterprise() {
EndpointConfiguration config = build(false);
assertTrue(config.getDisabledGroups().contains("enterprise"));
}
@Test
@DisplayName("pro build does not disable the enterprise group")
void proDoesNotDisableEnterprise() {
EndpointConfiguration config = build(true);
assertFalse(config.getDisabledGroups().contains("enterprise"));
}
}
@Nested
@DisplayName("getEndpointStatuses (Lombok getter) and logging summary")
class MiscTests {
@Test
@DisplayName("getEndpointStatuses reflects explicit disable state")
void endpointStatusesReflectDisable() {
EndpointConfiguration config = buildDefault();
config.disableEndpoint("merge-pdfs");
assertEquals(Boolean.FALSE, config.getEndpointStatuses().get("merge-pdfs"));
}
@Test
@DisplayName("logDisabledEndpointsSummary runs without throwing")
void logSummaryDoesNotThrow() {
EndpointConfiguration config = buildDefault();
config.disableGroup("PageOps");
config.disableGroup("qpdf");
// purely a smoke test of the logging branch coverage
config.logDisabledEndpointsSummary();
}
@Test
@DisplayName("logDisabledEndpointsSummary runs when nothing is disabled")
void logSummaryNothingDisabled() {
applicationProperties.getSystem().setEnableUrlToPDF(true);
EndpointConfiguration config = build(true);
config.logDisabledEndpointsSummary();
}
}
}
@@ -1,153 +0,0 @@
package stirling.software.SPDF.pdf.parser;
import static org.assertj.core.api.Assertions.assertThat;
import static stirling.software.SPDF.pdf.parser.PdfModels.*;
import java.util.List;
import org.junit.jupiter.api.Test;
/**
* Unit tests for {@link LineAlignmentTableParser}, focused on the coincident-line merge logic and
* column-grid construction.
*/
class LineAlignmentTableParserTest {
private final LineAlignmentTableParser parser = new LineAlignmentTableParser();
// ── mergeCoincidentLines ─────────────────────────────────────────────────────────────────────
@Test
void mergeCoincidentLines_singleLine_unchanged() {
var lines = List.of(tokenized(rawLine(10f, 100f, "Revenue")));
assertThat(parser.mergeCoincidentLines(lines)).hasSize(1);
}
@Test
void mergeCoincidentLines_distinctYLines_unchanged() {
// Two lines at different y positions — must NOT be merged.
var lines =
List.of(
tokenized(rawLine(10f, 100f, "Revenue")),
tokenized(rawLine(10f, 115f, "Cost")));
assertThat(parser.mergeCoincidentLines(lines)).hasSize(2);
}
@Test
void mergeCoincidentLines_sameY_merged() {
// Simulates a financial-table row split by LineBuilder at the column gap:
// label fragment at x=72 → "Revenue"
// value fragment at x=350 → "1,234"
// Both have y=100. After merge they should form one TokenizedLine.
var label = rawLine(72f, 100f, "Revenue");
var value = rawLine(350f, 100f, "1,234");
var merged = parser.mergeCoincidentLines(List.of(tokenized(label), tokenized(value)));
assertThat(merged).hasSize(1);
// The merged line should contain tokens from both halves.
var tokens = merged.get(0).all();
assertThat(tokens.stream().map(t -> t.text()).toList())
.containsExactlyInAnyOrder("Revenue", "1,234");
}
@Test
void mergeCoincidentLines_sameY_mergedLineHasCorrectBounds() {
var label = rawLine(72f, 100f, "Revenue"); // 7 chars × 6pt = 42pt wide → right = 114
var value = rawLine(350f, 100f, "1,234"); // 5 chars × 6pt = 30pt wide → right = 380
var merged = parser.mergeCoincidentLines(List.of(tokenized(label), tokenized(value)));
var bounds = merged.get(0).line().bounds();
assertThat(bounds.x()).isEqualTo(72f);
assertThat(bounds.right()).isEqualTo(380f);
}
@Test
void mergeCoincidentLines_withinTolerance_merged() {
// Lines 1.5pt apart (within ROW_MERGE_TOLERANCE_PT = 2pt) should merge.
var a = rawLine(10f, 100.0f, "Alpha");
var b = rawLine(200f, 101.5f, "99");
var merged = parser.mergeCoincidentLines(List.of(tokenized(a), tokenized(b)));
assertThat(merged).hasSize(1);
}
@Test
void mergeCoincidentLines_beyondTolerance_notMerged() {
// Lines 3pt apart (beyond ROW_MERGE_TOLERANCE_PT = 2pt) should NOT merge.
var a = rawLine(10f, 100.0f, "Alpha");
var b = rawLine(200f, 103.0f, "99");
var merged = parser.mergeCoincidentLines(List.of(tokenized(a), tokenized(b)));
assertThat(merged).hasSize(2);
}
@Test
void mergeCoincidentLines_threeCoincident_allMerged() {
// Three fragments at the same y (e.g. wide financial table with two value columns).
var a = rawLine(72f, 100f, "Revenue");
var b = rawLine(300f, 100f, "1,234");
var c = rawLine(400f, 100f, "5,678");
var merged = parser.mergeCoincidentLines(List.of(tokenized(a), tokenized(b), tokenized(c)));
assertThat(merged).hasSize(1);
assertThat(merged.get(0).all()).hasSize(3);
}
@Test
void mergeCoincidentLines_coincidentPairFollowedByDistinctLine_twoGroups() {
var a = rawLine(72f, 100f, "Revenue");
var b = rawLine(350f, 100f, "1,234"); // same y as a → merges with a
var c = rawLine(10f, 115f, "Expenses"); // different y → stays separate
var merged = parser.mergeCoincidentLines(List.of(tokenized(a), tokenized(b), tokenized(c)));
assertThat(merged).hasSize(2);
}
@Test
void mergeCoincidentLines_numericAnchorStatus_correctAfterMerge() {
// After merging, the combined line should be an anchor (≥2 numeric tokens).
// "Revenue" alone → not an anchor. "1,234 567" alone → anchor.
// Merged → anchor with at least 2 numerics.
var label = rawLine(72f, 100f, "Revenue");
var values = rawLineMultiWord(350f, 100f, "1,234", 30f, "567", 30f);
var merged = parser.mergeCoincidentLines(List.of(tokenized(label), tokenized(values)));
assertThat(merged).hasSize(1);
assertThat(merged.get(0).isAnchor()).isTrue();
}
// ── helpers ──────────────────────────────────────────────────────────────────────────────────
/** Creates a RawLine with a single TextFragment of the given text at the given position. */
private static RawLine rawLine(float x, float y, String text) {
float width = text.length() * 6f; // ~6pt per char — rough but consistent
float height = 12f;
Bounds bounds = new Bounds(x, y, width, height);
TextFragment fragment =
new TextFragment("tf-test", text, bounds, y + height, 11f, "Helvetica", false);
return new RawLine("ln-test", List.of(fragment), bounds, 1);
}
/**
* Creates a RawLine with two TextFragments representing two words separated by a small gap.
* Used to simulate a values-only line with multiple numeric tokens.
*/
private static RawLine rawLineMultiWord(
float x, float y, String word1, float w1, String word2, float w2) {
float height = 12f;
Bounds b1 = new Bounds(x, y, w1, height);
Bounds b2 = new Bounds(x + w1 + 5f, y, w2, height);
TextFragment f1 = new TextFragment("tf-1", word1, b1, y + height, 11f, "Helvetica", false);
TextFragment f2 = new TextFragment("tf-2", word2, b2, y + height, 11f, "Helvetica", false);
Bounds lineBounds = new Bounds(x, y, x + w1 + 5f + w2 - x, height);
return new RawLine("ln-test", List.of(f1, f2), lineBounds, 1);
}
/** Tokenises a RawLine via the parser's own tokenise logic (package-private access). */
private LineAlignmentTableParser.TokenizedLine tokenized(RawLine line) {
return parser.tokenize(line);
}
}
@@ -0,0 +1,345 @@
package stirling.software.SPDF.pdf.parser;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertSame;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static stirling.software.SPDF.pdf.parser.PdfModels.RawPage;
import static stirling.software.SPDF.pdf.parser.PdfModels.TableCell;
import static stirling.software.SPDF.pdf.parser.PdfModels.TableFragment;
import static stirling.software.SPDF.pdf.parser.PdfModels.TableRow;
import java.awt.Color;
import java.io.ByteArrayOutputStream;
import java.util.List;
import org.apache.pdfbox.Loader;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDPage;
import org.apache.pdfbox.pdmodel.PDPageContentStream;
import org.apache.pdfbox.pdmodel.common.PDRectangle;
import org.apache.pdfbox.pdmodel.font.PDType1Font;
import org.apache.pdfbox.pdmodel.font.Standard14Fonts;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test;
/**
* Unit tests for {@link TabulaTableParser}. Tables are built in-memory with PDFBox so the tests are
* deterministic and need no fixtures, network, or external processes.
*/
class TabulaTableParserGapTest {
private final TabulaTableParser parser = new TabulaTableParser();
// ── error / empty branches ───────────────────────────────────────────────
@Nested
@DisplayName("Empty and error branches")
class EmptyAndErrorBranches {
@Test
@DisplayName("page number 0 is out of Tabula's 1-based range -> empty list, no throw")
void pageNumberZeroReturnsEmpty() throws Exception {
byte[] pdf = pdfWithText(new String[] {"hello"});
try (PDDocument doc = Loader.loadPDF(pdf)) {
List<TableFragment> result = parser.parse(doc, 0);
assertNotNull(result);
assertTrue(result.isEmpty());
}
}
@Test
@DisplayName("page number beyond the document -> empty list, exception swallowed")
void pageNumberOutOfRangeReturnsEmpty() throws Exception {
byte[] pdf = pdfWithText(new String[] {"hello"});
try (PDDocument doc = Loader.loadPDF(pdf)) {
List<TableFragment> result = parser.parse(doc, 99);
assertNotNull(result);
assertTrue(result.isEmpty());
}
}
@Test
@DisplayName("negative page number -> empty list")
void negativePageNumberReturnsEmpty() throws Exception {
byte[] pdf = pdfWithText(new String[] {"hello"});
try (PDDocument doc = Loader.loadPDF(pdf)) {
assertTrue(parser.parse(doc, -5).isEmpty());
}
}
@Test
@DisplayName("lattice mode on a page with no ruled lines -> no tables")
void latticeWithNoRulingsReturnsEmpty() throws Exception {
byte[] pdf = pdfWithText(new String[] {"just some prose", "no table here"});
try (PDDocument doc = Loader.loadPDF(pdf)) {
List<TableFragment> result = parser.parse(doc, new RawPage(1, 0f, 0f, List.of()));
assertNotNull(result);
assertTrue(
result.isEmpty(), "borderless text must not be detected in lattice mode");
}
}
@Test
@DisplayName("blank page in lattice mode -> empty list")
void blankPageLatticeReturnsEmpty() throws Exception {
byte[] pdf = blankPdf();
try (PDDocument doc = Loader.loadPDF(pdf)) {
assertTrue(parser.parse(doc, new RawPage(1, 0f, 0f, List.of())).isEmpty());
}
}
}
// ── stream mode (BasicExtractionAlgorithm) ───────────────────────────────
@Nested
@DisplayName("Stream mode")
class StreamMode {
@Test
@DisplayName("page with text yields at least one well-formed fragment")
void streamOnTextProducesFragment() throws Exception {
byte[] pdf =
pdfWithText(new String[] {"Name Age City", "Alice 30 Paris", "Bob 25 Rome"});
try (PDDocument doc = Loader.loadPDF(pdf)) {
List<TableFragment> fragments =
parser.parseStream(doc, new RawPage(1, 0f, 0f, List.of()));
assertNotNull(fragments);
assertFalse(fragments.isEmpty(), "stream mode always builds a table from text");
assertFragmentWellFormed(fragments.get(0), 1, 0);
}
}
@Test
@DisplayName("fragment ids encode page and index")
void streamFragmentIdFormat() throws Exception {
byte[] pdf = pdfWithText(new String[] {"col1 col2", "a b"});
try (PDDocument doc = Loader.loadPDF(pdf)) {
List<TableFragment> fragments =
parser.parseStream(doc, new RawPage(1, 0f, 0f, List.of()));
assertFalse(fragments.isEmpty());
assertEquals("tbl-p1-0", fragments.get(0).tableId());
assertEquals(1, fragments.get(0).pageNumber());
}
}
@Test
@DisplayName("rawRows and the parsed rows stay in lockstep")
void streamRowsMatchRawRows() throws Exception {
byte[] pdf = pdfWithText(new String[] {"x y", "1 2", "3 4"});
try (PDDocument doc = Loader.loadPDF(pdf)) {
List<TableFragment> fragments =
parser.parseStream(doc, new RawPage(1, 0f, 0f, List.of()));
assertFalse(fragments.isEmpty());
TableFragment f = fragments.get(0);
assertEquals(f.rawRows().size(), f.rows().size());
}
}
}
// ── lattice mode with a real bordered grid ───────────────────────────────
@Nested
@DisplayName("Lattice mode")
class LatticeMode {
@Test
@DisplayName("bordered grid is detected and produces well-formed fragments")
void latticeDetectsBorderedTable() throws Exception {
byte[] pdf = pdfWithGrid();
try (PDDocument doc = Loader.loadPDF(pdf)) {
List<TableFragment> fragments =
parser.parse(doc, new RawPage(1, 0f, 0f, List.of()));
assertNotNull(fragments);
assertFalse(
fragments.isEmpty(), "a clean ruled grid must be detected in lattice mode");
TableFragment f = fragments.get(0);
assertFragmentWellFormed(f, 1, 0);
assertTrue(f.columnCount() >= 1, "a detected grid must have at least one column");
assertFalse(f.rawRows().isEmpty(), "a detected grid must have rows");
}
}
@Test
@DisplayName("convenience overload with page number routes to lattice mode")
void parseByPageNumberDetectsGrid() throws Exception {
byte[] pdf = pdfWithGrid();
try (PDDocument doc = Loader.loadPDF(pdf)) {
List<TableFragment> fragments = parser.parse(doc, 1);
assertNotNull(fragments);
assertFalse(fragments.isEmpty());
assertEquals(1, fragments.get(0).pageNumber());
}
}
@Test
@DisplayName("cell text is normalised (trimmed, newlines collapsed)")
void latticeCellTextIsNormalised() throws Exception {
byte[] pdf = pdfWithGrid();
try (PDDocument doc = Loader.loadPDF(pdf)) {
List<TableFragment> fragments =
parser.parse(doc, new RawPage(1, 0f, 0f, List.of()));
assertFalse(fragments.isEmpty());
for (List<String> row : fragments.get(0).rawRows()) {
for (String cell : row) {
assertNotNull(cell);
assertFalse(cell.contains("\n"), "newlines must be collapsed");
assertFalse(cell.contains("\r"), "carriage returns must be collapsed");
assertEquals(cell.trim(), cell, "cell text must be trimmed");
}
}
}
}
}
// ── contract invariants ──────────────────────────────────────────────────
@Nested
@DisplayName("Contract invariants")
class ContractInvariants {
@Test
@DisplayName("parse never returns null")
void parseNeverReturnsNull() throws Exception {
byte[] pdf = pdfWithText(new String[] {"abc"});
try (PDDocument doc = Loader.loadPDF(pdf)) {
assertNotNull(parser.parse(doc, new RawPage(1, 0f, 0f, List.of())));
assertNotNull(parser.parse(doc, 1));
assertNotNull(parser.parseStream(doc, new RawPage(1, 0f, 0f, List.of())));
}
}
@Test
@DisplayName("the document is not closed by the parser")
void documentRemainsOpenAfterParse() throws Exception {
byte[] pdf = pdfWithText(new String[] {"keep me open"});
try (PDDocument doc = Loader.loadPDF(pdf)) {
parser.parse(doc, new RawPage(1, 0f, 0f, List.of()));
parser.parseStream(doc, new RawPage(1, 0f, 0f, List.of()));
// ObjectExtractor.close() would close the underlying COSDocument; the parser must
// not.
assertFalse(
doc.getDocument().isClosed(),
"parser must not close the caller's document");
assertEquals(1, doc.getNumberOfPages());
}
}
}
// ── helpers ──────────────────────────────────────────────────────────────
/** Asserts every field of a fragment satisfies the documented contract. */
private static void assertFragmentWellFormed(
TableFragment f, int expectedPage, int expectedIndex) {
assertNotNull(f);
assertEquals(expectedPage, f.pageNumber());
assertEquals("tbl-p" + expectedPage + "-" + expectedIndex, f.tableId());
assertNotNull(f.bounds());
assertNotNull(f.headers());
assertTrue(f.headers().isEmpty(), "headers are deferred to v2 and must be empty");
assertNotNull(f.rows());
assertNotNull(f.rawRows());
assertNotNull(f.warnings());
assertSame(null, f.continuedFromPage(), "continuedFromPage is deferred to v2");
assertTrue(f.columnCount() >= 0);
assertTrue(f.confidence() >= 0f && f.confidence() <= 1f, "confidence must be within [0,1]");
assertEquals(f.rawRows().size(), f.rows().size());
for (TableRow row : f.rows()) {
assertNotNull(row.cells());
for (TableCell cell : row.cells()) {
assertNotNull(cell.text());
assertNotNull(cell.bounds());
assertEquals(1, cell.colSpan(), "colSpan is always 1 in v1");
assertEquals(1, cell.rowSpan(), "rowSpan is always 1 in v1");
}
}
}
private static byte[] pdfWithText(String[] lines) throws Exception {
try (PDDocument doc = new PDDocument()) {
PDPage page = new PDPage(PDRectangle.A4);
doc.addPage(page);
try (PDPageContentStream cs = new PDPageContentStream(doc, page)) {
cs.setFont(new PDType1Font(Standard14Fonts.FontName.HELVETICA), 12);
cs.setNonStrokingColor(Color.BLACK);
float y = 720f;
for (String line : lines) {
cs.beginText();
cs.newLineAtOffset(72f, y);
cs.showText(line);
cs.endText();
y -= 20f;
}
}
return save(doc);
}
}
private static byte[] blankPdf() throws Exception {
try (PDDocument doc = new PDDocument()) {
doc.addPage(new PDPage(PDRectangle.A4));
return save(doc);
}
}
/**
* Builds a small 3-row x 3-column ruled grid with text in each cell. The ruled lines make the
* table detectable by lattice mode.
*/
private static byte[] pdfWithGrid() throws Exception {
try (PDDocument doc = new PDDocument()) {
PDPage page = new PDPage(PDRectangle.A4);
doc.addPage(page);
float left = 100f;
float right = 400f;
float top = 700f;
float bottom = 550f;
int cols = 3;
int rows = 3;
float colStep = (right - left) / cols;
float rowStep = (top - bottom) / rows;
try (PDPageContentStream cs = new PDPageContentStream(doc, page)) {
cs.setStrokingColor(Color.BLACK);
cs.setLineWidth(1f);
// vertical lines
for (int c = 0; c <= cols; c++) {
float x = left + c * colStep;
cs.moveTo(x, bottom);
cs.lineTo(x, top);
}
// horizontal lines
for (int r = 0; r <= rows; r++) {
float yLine = bottom + r * rowStep;
cs.moveTo(left, yLine);
cs.lineTo(right, yLine);
}
cs.stroke();
// cell text
cs.setFont(new PDType1Font(Standard14Fonts.FontName.HELVETICA), 10);
cs.setNonStrokingColor(Color.BLACK);
for (int r = 0; r < rows; r++) {
for (int c = 0; c < cols; c++) {
cs.beginText();
cs.newLineAtOffset(left + c * colStep + 5f, top - (r + 1) * rowStep + 6f);
cs.showText("R" + r + "C" + c);
cs.endText();
}
}
}
return save(doc);
}
}
private static byte[] save(PDDocument doc) throws Exception {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
doc.save(baos);
return baos.toByteArray();
}
}
@@ -3,11 +3,13 @@ package stirling.software.common.cluster.inprocess;
import static org.junit.jupiter.api.Assertions.assertArrayEquals;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import org.junit.jupiter.api.Test;
@@ -39,4 +41,46 @@ class LocalDiskFileStoreTest {
assertThrows(IllegalArgumentException.class, () -> store.resolve("a/b"));
assertThrows(IllegalArgumentException.class, () -> store.resolve("a\\b"));
}
@Test
void ownerSidecarCannotBeReadAsFileId(@TempDir Path dir) throws IOException {
LocalDiskFileStore store = new LocalDiskFileStore(dir.toString());
FileStore.Stored stored =
store.store(new ByteArrayInputStream("hi".getBytes()), "f.bin", "alice");
String sidecarId = stored.fileId() + ".owner";
assertThrows(IllegalArgumentException.class, () -> store.resolve(sidecarId));
assertThrows(IllegalArgumentException.class, () -> store.retrieveBytes(sidecarId));
}
@Test
void ownerIsPersistedAndReturnedByGetOwner(@TempDir Path dir) throws IOException {
LocalDiskFileStore store = new LocalDiskFileStore(dir.toString());
FileStore.Stored stored =
store.store(new ByteArrayInputStream("hi".getBytes()), "f.bin", "alice");
assertEquals("alice", store.getOwner(stored.fileId()));
}
@Test
void getOwnerReturnsNullWhenNoOwnerWasRecorded(@TempDir Path dir) throws IOException {
LocalDiskFileStore store = new LocalDiskFileStore(dir.toString());
FileStore.Stored stored =
store.store(new ByteArrayInputStream("hi".getBytes()), "f.bin", null);
assertNull(store.getOwner(stored.fileId()));
}
@Test
void getOwnerReturnsNullForUnknownFileId(@TempDir Path dir) throws IOException {
LocalDiskFileStore store = new LocalDiskFileStore(dir.toString());
assertNull(store.getOwner("00000000-0000-0000-0000-000000000000"));
}
@Test
void deleteRemovesOwnerSidecar(@TempDir Path dir) throws IOException {
LocalDiskFileStore store = new LocalDiskFileStore(dir.toString());
FileStore.Stored stored =
store.store(new ByteArrayInputStream("hi".getBytes()), "f.bin", "alice");
assertTrue(store.delete(stored.fileId()));
assertFalse(Files.exists(dir.resolve(stored.fileId() + ".owner")));
assertNull(store.getOwner(stored.fileId()));
}
}
@@ -0,0 +1,520 @@
package stirling.software.common.configuration;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertSame;
import static org.junit.jupiter.api.Assertions.assertTrue;
import java.nio.file.Path;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.model.ApplicationProperties.CustomPaths.Operations;
import stirling.software.common.model.ApplicationProperties.CustomPaths.Pipeline;
import stirling.software.common.model.ApplicationProperties.ProcessExecutor.UnoServerEndpoint;
/**
* Unit tests for {@link RuntimePathConfig}. All of the resolution logic lives in the constructor,
* so each test builds a real {@link ApplicationProperties} (a plain @Data POJO with sensible
* defaults), constructs the config, and asserts on the exposed getters.
*/
class RuntimePathConfigTest {
/** The base path the production code derives from {@link InstallationPathConfig#getPath()}. */
private static final String BASE_PATH = InstallationPathConfig.getPath();
private static ApplicationProperties newProperties() {
return new ApplicationProperties();
}
private static RuntimePathConfig build(ApplicationProperties properties) {
return new RuntimePathConfig(properties);
}
@Nested
@DisplayName("Pipeline directory resolution")
class PipelinePaths {
@Test
@DisplayName("Defaults to <basePath>/pipeline and derived sub-folders")
void defaultPipelinePaths() {
RuntimePathConfig config = build(newProperties());
String expectedPipeline = Path.of(BASE_PATH, "pipeline").toString();
assertEquals(expectedPipeline, config.getPipelinePath());
// Watched folders are resolved to an absolute, normalized path by the production code.
assertEquals(
Path.of(expectedPipeline, "watchedFolders")
.toAbsolutePath()
.normalize()
.toString(),
config.getPipelineWatchedFoldersPath());
assertEquals(
Path.of(expectedPipeline, "finishedFolders").toString(),
config.getPipelineFinishedFoldersPath());
assertEquals(
Path.of(expectedPipeline, "defaultWebUIConfigs").toString(),
config.getPipelineDefaultWebUiConfigs());
}
@Test
@DisplayName("Custom pipelineDir overrides the default pipeline path")
void customPipelineDir() {
ApplicationProperties properties = newProperties();
Pipeline pipeline = properties.getSystem().getCustomPaths().getPipeline();
pipeline.setPipelineDir("/custom/pipeline");
RuntimePathConfig config = build(properties);
assertEquals("/custom/pipeline", config.getPipelinePath());
// Sub-folders are derived from the (already-resolved) custom pipeline path.
assertEquals(
Path.of("/custom/pipeline", "finishedFolders").toString(),
config.getPipelineFinishedFoldersPath());
assertEquals(
Path.of("/custom/pipeline", "defaultWebUIConfigs").toString(),
config.getPipelineDefaultWebUiConfigs());
}
@Test
@DisplayName("Blank pipelineDir falls back to the default")
void blankPipelineDirFallsBackToDefault() {
ApplicationProperties properties = newProperties();
properties.getSystem().getCustomPaths().getPipeline().setPipelineDir(" ");
RuntimePathConfig config = build(properties);
assertEquals(Path.of(BASE_PATH, "pipeline").toString(), config.getPipelinePath());
}
@Test
@DisplayName("Custom finished and webUI configs dirs override defaults")
void customFinishedAndWebUiDirs() {
ApplicationProperties properties = newProperties();
Pipeline pipeline = properties.getSystem().getCustomPaths().getPipeline();
pipeline.setFinishedFoldersDir("/custom/finished");
pipeline.setWebUIConfigsDir("/custom/webui");
RuntimePathConfig config = build(properties);
assertEquals("/custom/finished", config.getPipelineFinishedFoldersPath());
assertEquals("/custom/webui", config.getPipelineDefaultWebUiConfigs());
}
}
@Nested
@DisplayName("Watched folder resolution")
class WatchedFolders {
@Test
@DisplayName("Default watched folder is <pipeline>/watchedFolders and list has one entry")
void defaultWatchedFolder() {
RuntimePathConfig config = build(newProperties());
// Watched folders are resolved to an absolute, normalized path by the production code.
String expected =
Path.of(Path.of(BASE_PATH, "pipeline").toString(), "watchedFolders")
.toAbsolutePath()
.normalize()
.toString();
assertEquals(expected, config.getPipelineWatchedFoldersPath());
assertEquals(1, config.getPipelineWatchedFoldersPaths().size());
assertEquals(expected, config.getPipelineWatchedFoldersPaths().get(0));
}
@Test
@DisplayName("Legacy single watchedFoldersDir is used when no list is provided")
void legacyWatchedFolder() {
ApplicationProperties properties = newProperties();
properties
.getSystem()
.getCustomPaths()
.getPipeline()
.setWatchedFoldersDir("relativeWatched");
RuntimePathConfig config = build(properties);
// Legacy paths are normalized to absolute.
String expected = Path.of("relativeWatched").toAbsolutePath().normalize().toString();
assertEquals(1, config.getPipelineWatchedFoldersPaths().size());
assertEquals(expected, config.getPipelineWatchedFoldersPath());
}
@Test
@DisplayName("New list config takes precedence over the legacy single dir")
void listTakesPrecedenceOverLegacy() {
ApplicationProperties properties = newProperties();
Pipeline pipeline = properties.getSystem().getCustomPaths().getPipeline();
pipeline.setWatchedFoldersDir("legacyDir");
pipeline.setWatchedFoldersDirs(new ArrayList<>(Arrays.asList("listDirA", "listDirB")));
RuntimePathConfig config = build(properties);
List<String> paths = config.getPipelineWatchedFoldersPaths();
assertEquals(2, paths.size());
assertEquals(Path.of("listDirA").toAbsolutePath().normalize().toString(), paths.get(0));
assertEquals(Path.of("listDirB").toAbsolutePath().normalize().toString(), paths.get(1));
// The legacy value must NOT appear when the list is present.
assertFalse(
paths.contains(Path.of("legacyDir").toAbsolutePath().normalize().toString()));
}
@Test
@DisplayName("Duplicate paths in the list are de-duplicated after normalization")
void duplicatePathsAreDeduplicated() {
ApplicationProperties properties = newProperties();
properties
.getSystem()
.getCustomPaths()
.getPipeline()
.setWatchedFoldersDirs(
new ArrayList<>(Arrays.asList("dupDir", "dupDir", "otherDir")));
RuntimePathConfig config = build(properties);
List<String> paths = config.getPipelineWatchedFoldersPaths();
assertEquals(2, paths.size());
assertEquals(Path.of("dupDir").toAbsolutePath().normalize().toString(), paths.get(0));
assertEquals(Path.of("otherDir").toAbsolutePath().normalize().toString(), paths.get(1));
}
@Test
@DisplayName("Blank and whitespace-only list entries are sanitized out")
void blankListEntriesAreFiltered() {
ApplicationProperties properties = newProperties();
properties
.getSystem()
.getCustomPaths()
.getPipeline()
.setWatchedFoldersDirs(
new ArrayList<>(Arrays.asList(" ", "", "validDir", " ")));
RuntimePathConfig config = build(properties);
List<String> paths = config.getPipelineWatchedFoldersPaths();
assertEquals(1, paths.size());
assertEquals(Path.of("validDir").toAbsolutePath().normalize().toString(), paths.get(0));
}
@Test
@DisplayName("List entries are trimmed before resolution")
void listEntriesAreTrimmed() {
ApplicationProperties properties = newProperties();
properties
.getSystem()
.getCustomPaths()
.getPipeline()
.setWatchedFoldersDirs(new ArrayList<>(Arrays.asList(" spacedDir ")));
RuntimePathConfig config = build(properties);
assertEquals(
Path.of("spacedDir").toAbsolutePath().normalize().toString(),
config.getPipelineWatchedFoldersPath());
}
@Test
@DisplayName("An all-blank list falls back to the legacy dir, then default")
void allBlankListFallsBackToDefault() {
ApplicationProperties properties = newProperties();
properties
.getSystem()
.getCustomPaths()
.getPipeline()
.setWatchedFoldersDirs(new ArrayList<>(Arrays.asList("", " ")));
RuntimePathConfig config = build(properties);
// sanitizePathList strips everything -> empty -> falls through to default watched
// folder.
// The default is also resolved to an absolute, normalized path by the production code.
String expectedDefault =
Path.of(Path.of(BASE_PATH, "pipeline").toString(), "watchedFolders")
.toAbsolutePath()
.normalize()
.toString();
assertEquals(1, config.getPipelineWatchedFoldersPaths().size());
assertEquals(expectedDefault, config.getPipelineWatchedFoldersPath());
}
@Test
@DisplayName("First watched folder path is always exposed via the singular getter")
void singularGetterReturnsFirstEntry() {
ApplicationProperties properties = newProperties();
properties
.getSystem()
.getCustomPaths()
.getPipeline()
.setWatchedFoldersDirs(new ArrayList<>(Arrays.asList("firstDir", "secondDir")));
RuntimePathConfig config = build(properties);
assertEquals(
config.getPipelineWatchedFoldersPaths().get(0),
config.getPipelineWatchedFoldersPath());
assertEquals(
Path.of("firstDir").toAbsolutePath().normalize().toString(),
config.getPipelineWatchedFoldersPath());
}
}
@Nested
@DisplayName("Operation tool path resolution")
class OperationPaths {
@Test
@DisplayName("Defaults to bare command names when not running in Docker")
void defaultOperationPaths() {
// The test host has no /.dockerenv, so the non-docker defaults apply.
RuntimePathConfig config = build(newProperties());
assertEquals("weasyprint", config.getWeasyPrintPath());
assertEquals("unoconvert", config.getUnoConvertPath());
assertEquals("ebook-convert", config.getCalibrePath());
assertEquals("ocrmypdf", config.getOcrMyPdfPath());
assertEquals("soffice", config.getSOfficePath());
}
@Test
@DisplayName("Custom operation paths override the defaults")
void customOperationPaths() {
ApplicationProperties properties = newProperties();
Operations operations = properties.getSystem().getCustomPaths().getOperations();
operations.setWeasyprint("/opt/custom/weasyprint");
operations.setUnoconvert("/opt/custom/unoconvert");
operations.setCalibre("/opt/custom/ebook-convert");
operations.setOcrmypdf("/opt/custom/ocrmypdf");
operations.setSoffice("/opt/custom/soffice");
RuntimePathConfig config = build(properties);
assertEquals("/opt/custom/weasyprint", config.getWeasyPrintPath());
assertEquals("/opt/custom/unoconvert", config.getUnoConvertPath());
assertEquals("/opt/custom/ebook-convert", config.getCalibrePath());
assertEquals("/opt/custom/ocrmypdf", config.getOcrMyPdfPath());
assertEquals("/opt/custom/soffice", config.getSOfficePath());
}
@Test
@DisplayName("Blank custom operation path falls back to the default")
void blankOperationPathFallsBack() {
ApplicationProperties properties = newProperties();
properties.getSystem().getCustomPaths().getOperations().setWeasyprint(" ");
RuntimePathConfig config = build(properties);
assertEquals("weasyprint", config.getWeasyPrintPath());
}
@Test
@DisplayName("A single custom path leaves the other operation paths at defaults")
void partialOperationOverride() {
ApplicationProperties properties = newProperties();
properties
.getSystem()
.getCustomPaths()
.getOperations()
.setSoffice("/usr/local/soffice");
RuntimePathConfig config = build(properties);
assertEquals("/usr/local/soffice", config.getSOfficePath());
assertEquals("weasyprint", config.getWeasyPrintPath());
assertEquals("unoconvert", config.getUnoConvertPath());
}
}
@Nested
@DisplayName("Tesseract data path resolution")
class TessdataPath {
@Test
@DisplayName("Explicit tessdataDir config wins over env var and default")
void configuredTessdataDirWins() {
ApplicationProperties properties = newProperties();
properties.getSystem().setTessdataDir("/my/tessdata");
RuntimePathConfig config = build(properties);
// Config setting has the highest priority regardless of TESSDATA_PREFIX env state.
assertEquals("/my/tessdata", config.getTessDataPath());
}
@Test
@DisplayName("tessDataPath is never null even with no config")
void tessDataPathNeverNull() {
RuntimePathConfig config = build(newProperties());
// With no config setting, the value comes from TESSDATA_PREFIX or the hard default,
// either of which is non-null.
assertNotNull(config.getTessDataPath());
assertFalse(config.getTessDataPath().isEmpty());
}
}
@Nested
@DisplayName("UNO server endpoint resolution")
class UnoServerEndpoints {
@Test
@DisplayName("Auto mode builds one endpoint when session limit is unset (defaults to 1)")
void autoSingleEndpointByDefault() {
// Default ApplicationProperties: autoUnoServer = true, libreOfficeSessionLimit = 0 ->
// 1.
RuntimePathConfig config = build(newProperties());
List<UnoServerEndpoint> endpoints = config.getUnoServerEndpoints();
assertEquals(1, endpoints.size());
assertEquals("127.0.0.1", endpoints.get(0).getHost());
assertEquals(2003, endpoints.get(0).getPort());
}
@Test
@DisplayName("Auto mode builds N endpoints on consecutive even ports")
void autoMultipleEndpoints() {
ApplicationProperties properties = newProperties();
properties.getProcessExecutor().getSessionLimit().setLibreOfficeSessionLimit(3);
RuntimePathConfig config = build(properties);
List<UnoServerEndpoint> endpoints = config.getUnoServerEndpoints();
assertEquals(3, endpoints.size());
assertEquals(2003, endpoints.get(0).getPort());
assertEquals(2005, endpoints.get(1).getPort());
assertEquals(2007, endpoints.get(2).getPort());
for (UnoServerEndpoint endpoint : endpoints) {
assertEquals("127.0.0.1", endpoint.getHost());
}
}
@Test
@DisplayName("Manual mode returns the configured (valid) endpoints")
void manualEndpointsAreUsed() {
ApplicationProperties properties = newProperties();
ApplicationProperties.ProcessExecutor processExecutor = properties.getProcessExecutor();
processExecutor.setAutoUnoServer(false);
UnoServerEndpoint endpoint = new UnoServerEndpoint();
endpoint.setHost("10.0.0.5");
endpoint.setPort(4000);
processExecutor.setUnoServerEndpoints(new ArrayList<>(Arrays.asList(endpoint)));
RuntimePathConfig config = build(properties);
List<UnoServerEndpoint> endpoints = config.getUnoServerEndpoints();
assertEquals(1, endpoints.size());
assertEquals("10.0.0.5", endpoints.get(0).getHost());
assertEquals(4000, endpoints.get(0).getPort());
}
@Test
@DisplayName("Manual mode filters out endpoints with blank host or non-positive port")
void manualEndpointsAreSanitized() {
ApplicationProperties properties = newProperties();
ApplicationProperties.ProcessExecutor processExecutor = properties.getProcessExecutor();
processExecutor.setAutoUnoServer(false);
UnoServerEndpoint valid = new UnoServerEndpoint();
valid.setHost("192.168.1.10");
valid.setPort(5000);
UnoServerEndpoint blankHost = new UnoServerEndpoint();
blankHost.setHost(" ");
blankHost.setPort(5001);
UnoServerEndpoint badPort = new UnoServerEndpoint();
badPort.setHost("192.168.1.11");
badPort.setPort(0);
processExecutor.setUnoServerEndpoints(
new ArrayList<>(Arrays.asList(valid, blankHost, badPort)));
RuntimePathConfig config = build(properties);
List<UnoServerEndpoint> endpoints = config.getUnoServerEndpoints();
assertEquals(1, endpoints.size());
assertEquals("192.168.1.10", endpoints.get(0).getHost());
assertEquals(5000, endpoints.get(0).getPort());
}
@Test
@DisplayName("Manual mode with no usable endpoints falls back to a single default endpoint")
void manualModeNoEndpointsFallsBackToDefault() {
ApplicationProperties properties = newProperties();
ApplicationProperties.ProcessExecutor processExecutor = properties.getProcessExecutor();
processExecutor.setAutoUnoServer(false);
processExecutor.setUnoServerEndpoints(new ArrayList<>());
RuntimePathConfig config = build(properties);
List<UnoServerEndpoint> endpoints = config.getUnoServerEndpoints();
assertEquals(1, endpoints.size());
assertEquals("127.0.0.1", endpoints.get(0).getHost());
assertEquals(2003, endpoints.get(0).getPort());
}
@Test
@DisplayName("Null processExecutor defaults to a single UNO endpoint")
void nullProcessExecutorDefaultsToSingleEndpoint() {
ApplicationProperties properties = newProperties();
properties.setProcessExecutor(null);
RuntimePathConfig config = build(properties);
List<UnoServerEndpoint> endpoints = config.getUnoServerEndpoints();
assertEquals(1, endpoints.size());
assertEquals("127.0.0.1", endpoints.get(0).getHost());
assertEquals(2003, endpoints.get(0).getPort());
}
}
@Nested
@DisplayName("General contract")
class GeneralContract {
@Test
@DisplayName("getProperties returns the same instance passed to the constructor")
void propertiesAccessorReturnsSameInstance() {
ApplicationProperties properties = newProperties();
RuntimePathConfig config = build(properties);
assertSame(properties, config.getProperties());
}
@Test
@DisplayName("basePath matches InstallationPathConfig.getPath()")
void basePathMatchesInstallationPath() {
RuntimePathConfig config = build(newProperties());
assertEquals(BASE_PATH, config.getBasePath());
}
@Test
@DisplayName("All resolved path getters are non-null")
void allPathsNonNull() {
RuntimePathConfig config = build(newProperties());
assertNotNull(config.getPipelinePath());
assertNotNull(config.getPipelineWatchedFoldersPath());
assertNotNull(config.getPipelineWatchedFoldersPaths());
assertNotNull(config.getPipelineFinishedFoldersPath());
assertNotNull(config.getPipelineDefaultWebUiConfigs());
assertNotNull(config.getWeasyPrintPath());
assertNotNull(config.getUnoConvertPath());
assertNotNull(config.getCalibrePath());
assertNotNull(config.getOcrMyPdfPath());
assertNotNull(config.getSOfficePath());
assertNotNull(config.getTessDataPath());
assertNotNull(config.getUnoServerEndpoints());
assertTrue(config.getUnoServerEndpoints().size() >= 1);
}
}
}
@@ -2,7 +2,7 @@ package stirling.software.common.model;
import static org.junit.jupiter.api.Assertions.*;
import java.nio.file.Paths;
import java.nio.file.Path;
import java.util.ArrayList;
import java.util.Collection;
import java.util.List;
@@ -31,18 +31,33 @@ class ApplicationPropertiesLogicTest {
assertTrue(sys.isAnalyticsEnabled());
}
@Test
void storageSigning_userListScope_defaultsToOrg_andIsSettable() {
// Self-host backward-compat: scope must default to "org" (saas profile pins "team").
ApplicationProperties.Storage.Signing signing = new ApplicationProperties.Storage.Signing();
assertFalse(signing.isEnabled());
assertEquals("org", signing.getUserListScope());
signing.setUserListScope("team");
assertEquals("team", signing.getUserListScope());
// Reachable from the full tree as storage.signing.userListScope.
assertEquals(
"org", new ApplicationProperties().getStorage().getSigning().getUserListScope());
}
@Test
void tempFileManagement_defaults_and_overrides() {
Function<String, String> normalize = s -> Paths.get(s).normalize().toString();
Function<String, String> normalize = s -> Path.of(s).normalize().toString();
ApplicationProperties.TempFileManagement tfm =
new ApplicationProperties.TempFileManagement();
String expectedBase =
Paths.get(java.lang.System.getProperty("java.io.tmpdir"), "stirling-pdf")
.toString();
Path.of(java.lang.System.getProperty("java.io.tmpdir"), "stirling-pdf").toString();
assertEquals(expectedBase, tfm.getBaseTmpDir());
String expectedLibre = Paths.get(expectedBase, "libreoffice").toString();
String expectedLibre = Path.of(expectedBase, "libreoffice").toString();
assertEquals(expectedLibre, tfm.getLibreofficeDir());
tfm.setBaseTmpDir("/custom/base");
@@ -0,0 +1,269 @@
package stirling.software.common.pdf;
import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.junit.jupiter.api.Assertions.fail;
import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.ArrayList;
import java.util.List;
import java.util.stream.Stream;
import org.junit.jupiter.api.Disabled;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.Arguments;
import org.junit.jupiter.params.provider.MethodSource;
import stirling.software.jpdfium.PdfDocument;
import stirling.software.jpdfium.text.TextLine;
import stirling.software.jpdfium.text.TextWord;
/**
* Accuracy and robustness tests for {@link PdfMarkdownConverter}, comparing conversion output
* against hand-authored golden Markdown for a set of owned/synthetic fixtures.
*
* <p>The {@link #gatedFixtures()} set is enforced in CI: those fixtures currently convert within
* the accuracy threshold and guard against regressions. Fixtures still being iterated on live in
* {@link #wipFixtures()} under a {@link Disabled} test so the goldens stay in the tree without
* breaking the build. Enable the WIP test locally to see per-fixture scores while working on the
* converter.
*/
class PdfMarkdownConverterTest {
/** Accuracy threshold: output must share at least this fraction of content with the golden. */
private static final double THRESHOLD = 0.95;
@TempDir Path tmp;
/** Fixtures that meet the accuracy threshold today and therefore gate CI. */
static Stream<Arguments> gatedFixtures() {
return Stream.of(
Arguments.of("multi-column-test_lorem.pdf", "multi-column-test_lorem.md"),
Arguments.of("bordered-table-test_widget.pdf", "bordered-table-test_widget.md"),
Arguments.of("many-tables-test_stress.pdf", "many-tables-test_stress.md"));
}
/** Fixtures still below the threshold; tracked here, enable locally to iterate. */
static Stream<Arguments> wipFixtures() {
return Stream.of(
Arguments.of(
"wrapped-cell-test_expense-report.pdf",
"wrapped-cell-test_expense-report.md"));
}
@ParameterizedTest(name = "{0}")
@MethodSource("gatedFixtures")
void convertMatchesGoldenMarkdown(String pdfName, String mdName) throws IOException {
assertConversionMatchesGolden(pdfName, mdName);
}
@Disabled("WIP fixtures below the accuracy threshold; enable locally to iterate")
@ParameterizedTest(name = "{0}")
@MethodSource("wipFixtures")
void convertMatchesGoldenMarkdownWip(String pdfName, String mdName) throws IOException {
assertConversionMatchesGolden(pdfName, mdName);
}
/**
* Degenerate/extreme geometry must not crash the converter. A crafted or malformed PDF can
* position text anywhere via a text matrix, so a row's words can span from near the origin to a
* coordinate beyond {@link Integer#MAX_VALUE}. The old column-detection code sized an {@code
* int[]} straight from {@code (int) Math.ceil(maxX) - lo}, which either allocated a multi-GB
* array (OutOfMemoryError) or overflowed to a negative length (NegativeArraySizeException) —
* taking down the request thread. Detection must instead bail out and return no columns.
*/
@Test
void columnDetectionSurvivesDegenerateGeometry() {
// x ≈ 2.5e9 is past Integer.MAX_VALUE; combined with a near-origin word it yields an
// implausible span that the pre-fix code turned into a fatal array allocation.
List<TextLine> rows = new ArrayList<>();
for (int r = 0; r < 4; r++) {
float y = 400f - r * 12f;
TextWord near = new TextWord(List.of(), 50f, y, 30f, 10f);
TextWord far = new TextWord(List.of(), 2_500_000_000f, y, 30f, 10f);
rows.add(new TextLine(List.of(near, far), 50f, y, 2_499_999_980f, 10f));
}
List<float[]> columns =
assertDoesNotThrow(() -> PdfMarkdownConverter.findColumnRangesFromLines(rows));
assertTrue(
columns.isEmpty(),
"implausible page span should disable column detection, not allocate from it");
}
private void assertConversionMatchesGolden(String pdfName, String mdName) throws IOException {
Path pdfPath = tmp.resolve(pdfName);
try (InputStream in =
getClass().getResourceAsStream("/pdf-ingestion-fixtures/" + pdfName)) {
if (in == null) {
fail("Fixture not found on classpath: /pdf-ingestion-fixtures/" + pdfName);
}
Files.copy(in, pdfPath);
}
String actual;
try (PdfDocument doc = PdfDocument.open(pdfPath)) {
actual = new PdfMarkdownConverter().convert(doc);
}
String expected;
try (InputStream in = getClass().getResourceAsStream("/pdf-ingestion-fixtures/" + mdName)) {
if (in == null) {
fail("Golden file not found on classpath: /pdf-ingestion-fixtures/" + mdName);
}
expected = new String(in.readAllBytes(), StandardCharsets.UTF_8);
}
// Image placeholders are not scored: their body text is a TODO ("ideally, add the info
// available about the image...") rather than real content, so comparing it would penalise
// output for matching a placeholder we intend to replace. Drop those lines from both sides.
expected = stripImagePlaceholders(expected);
actual = stripImagePlaceholders(actual);
double similarity = similarity(expected, actual);
if (similarity < THRESHOLD) {
fail(
String.format(
"Markdown output differs from golden file '%s' by %.1f%% (threshold %.0f%%):%n%s",
mdName,
(1.0 - similarity) * 100,
(1.0 - THRESHOLD) * 100,
unifiedDiff(expected, actual)));
}
}
/** Substring identifying an image-placeholder line, which is excluded from scoring. */
private static final String IMAGE_PLACEHOLDER_MARKER = "Image intentionally redacted";
/**
* Removes non-content lines from the comparison: image placeholders (TODO text we intend to
* replace) and GFM table separator rows (the {@code |---|---|} divider, whose exact dash count
* is cosmetic — any run of three or more dashes is valid Markdown).
*/
private static String stripImagePlaceholders(String md) {
StringBuilder sb = new StringBuilder();
for (String line : md.split("\n", -1)) {
if (line.contains(IMAGE_PLACEHOLDER_MARKER)
|| line.strip().startsWith("<image redacted")
|| isTableSeparatorRow(line)) {
continue;
}
if (sb.length() > 0) {
sb.append('\n');
}
sb.append(line);
}
return sb.toString();
}
/** True for a GFM table separator row, e.g. {@code |---|:--:|---|} (only |, -, :, space). */
private static boolean isTableSeparatorRow(String line) {
String t = line.strip();
if (!t.contains("-")) {
return false;
}
return t.chars().allMatch(c -> c == '|' || c == '-' || c == ':' || c == ' ');
}
/**
* Character-level similarity: proportion of expected characters that appear in the LCS. O(n*m)
* but golden files are small enough that this is fine.
*/
private static double similarity(String expected, String actual) {
if (expected.isEmpty() && actual.isEmpty()) return 1.0;
if (expected.isEmpty() || actual.isEmpty()) return 0.0;
// Strip all whitespace for a content-focused comparison
String e = expected.replaceAll("\\s+", " ").strip();
String a = actual.replaceAll("\\s+", " ").strip();
int lcs = lcsLength(e, a);
return (double) lcs / Math.max(e.length(), a.length());
}
private static int lcsLength(String a, String b) {
// Use two-row DP to keep memory reasonable
int m = a.length(), n = b.length();
int[] prev = new int[n + 1];
int[] curr = new int[n + 1];
for (int i = 1; i <= m; i++) {
for (int j = 1; j <= n; j++) {
if (a.charAt(i - 1) == b.charAt(j - 1)) {
curr[j] = prev[j - 1] + 1;
} else {
curr[j] = Math.max(curr[j - 1], prev[j]);
}
}
int[] tmp = prev;
prev = curr;
curr = tmp;
java.util.Arrays.fill(curr, 0);
}
return prev[n];
}
private static String unifiedDiff(String expected, String actual) {
String[] expectedLines = expected.split("\n", -1);
String[] actualLines = actual.split("\n", -1);
List<String> diff = new ArrayList<>();
diff.add("--- expected");
diff.add("+++ actual");
int maxLines = Math.max(expectedLines.length, actualLines.length);
int context = 3;
boolean inHunk = false;
int hunkStart = -1;
List<String> hunkLines = new ArrayList<>();
for (int i = 0; i < maxLines; i++) {
String exp = i < expectedLines.length ? expectedLines[i] : null;
String act = i < actualLines.length ? actualLines[i] : null;
boolean changed = exp == null || act == null || !exp.equals(act);
if (changed) {
if (!inHunk) {
inHunk = true;
hunkStart = Math.max(0, i - context);
// add context lines before change
for (int c = hunkStart; c < i; c++) {
hunkLines.add(" " + (c < expectedLines.length ? expectedLines[c] : ""));
}
}
if (exp != null) hunkLines.add("-" + exp);
if (act != null) hunkLines.add("+" + act);
} else {
if (inHunk) {
hunkLines.add(" " + exp);
// check if we're far enough past the last change to close the hunk
boolean moreChanges = false;
for (int j = i + 1; j < Math.min(i + context, maxLines); j++) {
String e2 = j < expectedLines.length ? expectedLines[j] : null;
String a2 = j < actualLines.length ? actualLines[j] : null;
if (e2 == null || a2 == null || !e2.equals(a2)) {
moreChanges = true;
break;
}
}
if (!moreChanges && (i - hunkStart) >= context) {
diff.add("@@ -" + (hunkStart + 1) + " @@");
diff.addAll(hunkLines);
hunkLines.clear();
inHunk = false;
}
}
}
}
if (inHunk && !hunkLines.isEmpty()) {
diff.add("@@ -" + (hunkStart + 1) + " @@");
diff.addAll(hunkLines);
}
return String.join("\n", diff);
}
}
@@ -5,6 +5,7 @@ import static org.mockito.Mockito.mock;
import java.io.IOException;
import java.nio.file.Path;
import java.util.Optional;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
@@ -19,7 +20,8 @@ class FileStorageDelegationTest {
FileStorage fs =
new FileStorage(
mock(FileOrUploadService.class),
new LocalDiskFileStore(tempDir.toString()));
new LocalDiskFileStore(tempDir.toString()),
Optional.empty());
byte[] payload = "round-trip".getBytes();
String id = fs.storeBytes(payload, "x.bin");
assertArrayEquals(payload, fs.retrieveBytes(id));
@@ -0,0 +1,107 @@
package stirling.software.common.service;
import static org.junit.jupiter.api.Assertions.assertArrayEquals;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
import java.io.IOException;
import java.nio.file.Path;
import java.util.Optional;
import java.util.concurrent.atomic.AtomicReference;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import stirling.software.common.cluster.inprocess.LocalDiskFileStore;
import stirling.software.common.util.JobContext;
class FileStorageOwnershipTest {
private FileStorage newStorageWithoutSecurity(Path tempDir) {
return new FileStorage(
mock(FileOrUploadService.class),
new LocalDiskFileStore(tempDir.toString()),
Optional.empty());
}
private FileStorage newStorageWithCurrentUser(Path tempDir, AtomicReference<String> userRef) {
JobOwnershipService svc = mock(JobOwnershipService.class);
when(svc.getCurrentUserId()).thenAnswer(invocation -> Optional.ofNullable(userRef.get()));
return new FileStorage(
mock(FileOrUploadService.class),
new LocalDiskFileStore(tempDir.toString()),
Optional.of(svc));
}
@Test
void desktopMode_noOwnershipService_storesAndRetrievesWithoutChecks(@TempDir Path tempDir)
throws IOException {
FileStorage fs = newStorageWithoutSecurity(tempDir);
byte[] payload = "desktop".getBytes();
String id = fs.storeBytes(payload, "x.bin");
assertArrayEquals(payload, fs.retrieveBytes(id));
}
@Test
void sameUserStoresAndRetrieves_allowed(@TempDir Path tempDir) throws IOException {
AtomicReference<String> user = new AtomicReference<>("alice");
FileStorage fs = newStorageWithCurrentUser(tempDir, user);
byte[] payload = "alice's file".getBytes();
String id = fs.storeBytes(payload, "x.bin");
assertArrayEquals(payload, fs.retrieveBytes(id));
}
@Test
void differentUserRetrieves_throwsSecurityException(@TempDir Path tempDir) throws IOException {
AtomicReference<String> user = new AtomicReference<>("alice");
FileStorage fs = newStorageWithCurrentUser(tempDir, user);
String id = fs.storeBytes("alice's file".getBytes(), "x.bin");
user.set("bob");
assertThrows(SecurityException.class, () -> fs.retrieveBytes(id));
assertThrows(SecurityException.class, () -> fs.retrieveInputStream(id));
assertThrows(SecurityException.class, () -> fs.getFileSize(id));
assertThrows(SecurityException.class, () -> fs.fileExists(id));
assertThrows(SecurityException.class, () -> fs.deleteFile(id));
}
@Test
void anonymousRetrieveOfOwnedFile_allowed_noCurrentUserMeansNoCompare(@TempDir Path tempDir)
throws IOException {
AtomicReference<String> user = new AtomicReference<>("alice");
FileStorage fs = newStorageWithCurrentUser(tempDir, user);
byte[] payload = "alice's file".getBytes();
String id = fs.storeBytes(payload, "x.bin");
user.set(null);
assertArrayEquals(payload, fs.retrieveBytes(id));
}
@Test
void authedRetrieveOfAnonymousFile_allowed_noOwnerOnFile(@TempDir Path tempDir)
throws IOException {
AtomicReference<String> user = new AtomicReference<>(null);
FileStorage fs = newStorageWithCurrentUser(tempDir, user);
byte[] payload = "no-owner".getBytes();
String id = fs.storeBytes(payload, "x.bin");
user.set("alice");
assertArrayEquals(payload, fs.retrieveBytes(id));
}
@Test
void propagatedOwner_scopesAsyncWriteWithNoLiveUser(@TempDir Path tempDir) throws IOException {
AtomicReference<String> user = new AtomicReference<>(null);
FileStorage fs = newStorageWithCurrentUser(tempDir, user);
byte[] payload = "alice's async result".getBytes();
String id;
try {
JobContext.setOwner("alice");
id = fs.storeBytes(payload, "x.bin");
} finally {
JobContext.clear();
}
user.set("alice");
assertArrayEquals(payload, fs.retrieveBytes(id));
user.set("bob");
assertThrows(SecurityException.class, () -> fs.retrieveBytes(id));
}
}
@@ -9,6 +9,8 @@ import java.io.InputStream;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.Optional;
import java.util.UUID;
import java.util.stream.Stream;
import org.junit.jupiter.api.BeforeEach;
@@ -37,7 +39,10 @@ class FileStorageTest {
void setUp() throws IOException {
MockitoAnnotations.openMocks(this);
fileStorage =
new FileStorage(fileOrUploadService, new LocalDiskFileStore(tempDir.toString()));
new FileStorage(
fileOrUploadService,
new LocalDiskFileStore(tempDir.toString()),
Optional.empty());
// Create a mock MultipartFile
mockFile = mock(MultipartFile.class);
@@ -79,7 +84,7 @@ class FileStorageTest {
void testRetrieveFile() throws IOException {
// Arrange
byte[] fileContent = "Test PDF content".getBytes();
String fileId = "test-file-1";
String fileId = UUID.randomUUID().toString();
Path filePath = tempDir.resolve(fileId);
Files.write(filePath, fileContent);
@@ -99,7 +104,7 @@ class FileStorageTest {
void testRetrieveBytes() throws IOException {
// Arrange
byte[] fileContent = "Test PDF content".getBytes();
String fileId = "test-file-2";
String fileId = UUID.randomUUID().toString();
Path filePath = tempDir.resolve(fileId);
Files.write(filePath, fileContent);
@@ -113,7 +118,7 @@ class FileStorageTest {
@Test
void testRetrieveFile_FileNotFound() {
// Arrange
String nonExistentFileId = "non-existent-file";
String nonExistentFileId = UUID.randomUUID().toString();
// Act & Assert
assertThrows(IOException.class, () -> fileStorage.retrieveFile(nonExistentFileId));
@@ -122,7 +127,7 @@ class FileStorageTest {
@Test
void testRetrieveBytes_FileNotFound() {
// Arrange
String nonExistentFileId = "non-existent-file";
String nonExistentFileId = UUID.randomUUID().toString();
// Act & Assert
assertThrows(IOException.class, () -> fileStorage.retrieveBytes(nonExistentFileId));
@@ -132,7 +137,7 @@ class FileStorageTest {
void testDeleteFile() throws IOException {
// Arrange
byte[] fileContent = "Test PDF content".getBytes();
String fileId = "test-file-3";
String fileId = UUID.randomUUID().toString();
Path filePath = tempDir.resolve(fileId);
Files.write(filePath, fileContent);
@@ -147,7 +152,7 @@ class FileStorageTest {
@Test
void testDeleteFile_FileNotFound() {
// Arrange
String nonExistentFileId = "non-existent-file";
String nonExistentFileId = UUID.randomUUID().toString();
// Act
boolean result = fileStorage.deleteFile(nonExistentFileId);
@@ -160,7 +165,7 @@ class FileStorageTest {
void testFileExists() throws IOException {
// Arrange
byte[] fileContent = "Test PDF content".getBytes();
String fileId = "test-file-4";
String fileId = UUID.randomUUID().toString();
Path filePath = tempDir.resolve(fileId);
Files.write(filePath, fileContent);
@@ -174,7 +179,7 @@ class FileStorageTest {
@Test
void testFileExists_FileNotFound() {
// Arrange
String nonExistentFileId = "non-existent-file";
String nonExistentFileId = UUID.randomUUID().toString();
// Act
boolean result = fileStorage.fileExists(nonExistentFileId);
@@ -59,6 +59,53 @@ class InternalApiClientTest {
servletContext, userService, tempFileManager, environment, applicationProperties);
}
@Test
void postTagsRequestAsAutomation() throws Exception {
// Every InternalApiClient.post() caller is a parent automation flow dispatching a child
// tool (pipeline executor, AI workflow, policy runner). Tagging the sub-step here means
// the saas PaygChargeInterceptor classifies it as BillingCategory.AUTOMATION regardless of
// the dispatched controller's @RequiresFeature — so an AI-OCR step inside a policy run
// bills as AUTOMATION, not AI. The header value is the literal string "true" because the
// interceptor compares case-insensitively-trimmed against that token.
MultiValueMap<String, Object> body = new LinkedMultiValueMap<>();
body.add("fileInput", namedResource("input.pdf", "data"));
Path tempPath = Files.createTempFile("internal-api-automation-test", ".tmp");
TempFile tempFile = mock(TempFile.class);
when(tempFile.getPath()).thenReturn(tempPath);
when(tempFile.getFile()).thenReturn(tempPath.toFile());
when(tempFileManager.createManagedTempFile("internal-api")).thenReturn(tempFile);
HttpHeaders[] captured = {null};
try (var ignored =
mockConstruction(
RestTemplate.class,
(rt, ctx) -> {
when(rt.httpEntityCallback(any(), eq(Resource.class)))
.thenAnswer(
inv -> {
HttpEntity<?> entity = inv.getArgument(0);
captured[0] = entity.getHeaders();
return (RequestCallback) req -> {};
});
when(rt.execute(anyString(), eq(HttpMethod.POST), any(), any()))
.thenAnswer(inv -> fakeOkResponse(inv.getArgument(3)));
})) {
InternalApiClient mockedClient = newClient();
mockedClient.post("/api/v1/general/merge-pdfs", body);
assertNotNull(captured[0]);
assertEquals(
"true",
captured[0].getFirst(InternalApiClient.AUTOMATION_HEADER),
"Sub-step dispatch must carry the automation marker header");
} finally {
Files.deleteIfExists(tempPath);
}
}
@Test
void postDoesNotForceContentType() throws Exception {
MultiValueMap<String, Object> body = new LinkedMultiValueMap<>();
@@ -0,0 +1,471 @@
package stirling.software.common.service;
import static org.junit.jupiter.api.Assertions.*;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.List;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import org.springframework.mock.web.MockMultipartFile;
import org.springframework.test.util.ReflectionTestUtils;
import org.springframework.web.multipart.MultipartFile;
import stirling.software.common.service.MobileScannerService.FileMetadata;
import stirling.software.common.service.MobileScannerService.SessionInfo;
/**
* Unit tests for {@link MobileScannerService}. The service stores uploaded files in a temp
* directory. To keep tests isolated and deterministic, the {@code tempDirectory} field is
* redirected to a JUnit {@link TempDir} via reflection after construction.
*/
class MobileScannerServiceTest {
@TempDir Path tempDir;
private MobileScannerService service;
@BeforeEach
void setUp() throws IOException {
service = new MobileScannerService();
// Redirect the service's temp directory to the isolated test temp dir.
ReflectionTestUtils.setField(service, "tempDirectory", tempDir);
}
private MultipartFile file(String name, String content) {
return new MockMultipartFile(
"file", name, "text/plain", content.getBytes(StandardCharsets.UTF_8));
}
private MultipartFile emptyFile(String name) {
return new MockMultipartFile("file", name, "text/plain", new byte[0]);
}
@Nested
@DisplayName("createSession")
class CreateSession {
@Test
@DisplayName("creates a session and returns coherent SessionInfo")
void createsSession() {
SessionInfo info = service.createSession("abc-123");
assertNotNull(info);
assertEquals("abc-123", info.getSessionId());
assertTrue(info.getCreatedAt() > 0);
assertEquals(10 * 60 * 1000L, info.getTimeoutMs());
assertEquals(info.getCreatedAt() + info.getTimeoutMs(), info.getExpiresAt());
}
@Test
@DisplayName("session is retrievable via validateSession after creation")
void createdSessionIsValid() {
service.createSession("sess1");
assertNotNull(service.validateSession("sess1"));
}
@Test
@DisplayName("rejects null session ID")
void rejectsNull() {
assertThrows(IllegalArgumentException.class, () -> service.createSession(null));
}
@Test
@DisplayName("rejects blank session ID")
void rejectsBlank() {
assertThrows(IllegalArgumentException.class, () -> service.createSession(" "));
}
@Test
@DisplayName("rejects session ID with invalid characters")
void rejectsInvalidChars() {
assertThrows(IllegalArgumentException.class, () -> service.createSession("bad/id"));
assertThrows(IllegalArgumentException.class, () -> service.createSession("bad id"));
assertThrows(IllegalArgumentException.class, () -> service.createSession("bad_id"));
}
@Test
@DisplayName("accepts alphanumeric and hyphen session IDs")
void acceptsValidChars() {
assertNotNull(service.createSession("ABC-def-123"));
}
}
@Nested
@DisplayName("validateSession")
class ValidateSession {
@Test
@DisplayName("returns null for unknown session")
void unknownReturnsNull() {
assertNull(service.validateSession("does-not-exist"));
}
@Test
@DisplayName("returns SessionInfo for an existing session")
void existingReturnsInfo() {
service.createSession("s1");
SessionInfo info = service.validateSession("s1");
assertNotNull(info);
assertEquals("s1", info.getSessionId());
assertEquals(10 * 60 * 1000L, info.getTimeoutMs());
}
@Test
@DisplayName("expires and removes a session whose last access is in the past")
void expiredSessionRemoved() {
service.createSession("expired");
// Force the underlying session's last access far into the past.
forceLastAccess("expired", System.currentTimeMillis() - (20 * 60 * 1000L));
assertNull(service.validateSession("expired"));
// After expiry the session should be gone entirely.
assertNull(service.validateSession("expired"));
}
}
@Nested
@DisplayName("uploadFiles")
class UploadFiles {
@Test
@DisplayName("stores files and records metadata")
void storesFiles() throws IOException {
service.createSession("up1");
service.uploadFiles("up1", List.of(file("scan.txt", "hello")));
List<FileMetadata> metas = service.getSessionFiles("up1");
assertEquals(1, metas.size());
FileMetadata meta = metas.get(0);
assertEquals("scan.txt", meta.getFilename());
assertEquals(5, meta.getSize());
assertEquals("text/plain", meta.getContentType());
// File physically exists on disk.
Path stored = tempDir.resolve("up1").resolve("scan.txt");
assertTrue(Files.exists(stored));
assertEquals("hello", Files.readString(stored));
}
@Test
@DisplayName("auto-creates a session when uploading to an unregistered session ID")
void autoCreatesSession() throws IOException {
service.uploadFiles("new-session", List.of(file("a.txt", "data")));
List<FileMetadata> metas = service.getSessionFiles("new-session");
assertEquals(1, metas.size());
}
@Test
@DisplayName("skips empty files")
void skipsEmptyFiles() throws IOException {
service.createSession("up2");
service.uploadFiles("up2", List.of(emptyFile("empty.txt"), file("real.txt", "x")));
List<FileMetadata> metas = service.getSessionFiles("up2");
assertEquals(1, metas.size());
assertEquals("real.txt", metas.get(0).getFilename());
}
@Test
@DisplayName("sanitizes dangerous filename characters")
void sanitizesFilename() throws IOException {
service.createSession("up3");
service.uploadFiles("up3", List.of(file("we ird@na#me.txt", "x")));
List<FileMetadata> metas = service.getSessionFiles("up3");
assertEquals(1, metas.size());
String stored = metas.get(0).getFilename();
// Disallowed chars replaced with underscores; allowed set is [a-zA-Z0-9._-].
assertTrue(stored.matches("[a-zA-Z0-9._-]+"), "unexpected filename: " + stored);
assertTrue(Files.exists(tempDir.resolve("up3").resolve(stored)));
}
@Test
@DisplayName("handles duplicate filenames by appending a counter")
void handlesDuplicateFilenames() throws IOException {
service.createSession("up4");
service.uploadFiles("up4", List.of(file("dup.txt", "one")));
service.uploadFiles("up4", List.of(file("dup.txt", "two")));
List<FileMetadata> metas = service.getSessionFiles("up4");
assertEquals(2, metas.size());
Path original = tempDir.resolve("up4").resolve("dup.txt");
Path renamed = tempDir.resolve("up4").resolve("dup-1.txt");
assertTrue(Files.exists(original));
assertTrue(Files.exists(renamed));
assertEquals("one", Files.readString(original));
assertEquals("two", Files.readString(renamed));
}
@Test
@DisplayName("falls back to a generated name when original filename is null")
void generatesNameWhenNull() throws IOException {
service.createSession("up5");
MultipartFile noName =
new MockMultipartFile("file", null, "text/plain", "x".getBytes());
service.uploadFiles("up5", List.of(noName));
List<FileMetadata> metas = service.getSessionFiles("up5");
assertEquals(1, metas.size());
assertTrue(metas.get(0).getFilename().startsWith("upload-"));
}
@Test
@DisplayName("rejects invalid session ID before any storage")
void rejectsInvalidSessionId() {
assertThrows(
IllegalArgumentException.class,
() -> service.uploadFiles("bad/id", List.of(file("a.txt", "x"))));
}
@Test
@DisplayName("uploading an empty list leaves no files")
void emptyListNoFiles() throws IOException {
service.createSession("up6");
service.uploadFiles("up6", List.of());
assertTrue(service.getSessionFiles("up6").isEmpty());
}
}
@Nested
@DisplayName("getSessionFiles")
class GetSessionFiles {
@Test
@DisplayName("returns empty list for unknown session")
void unknownReturnsEmpty() {
assertTrue(service.getSessionFiles("nope").isEmpty());
}
@Test
@DisplayName("returns a defensive copy of the metadata list")
void returnsDefensiveCopy() throws IOException {
service.createSession("g1");
service.uploadFiles("g1", List.of(file("a.txt", "x")));
List<FileMetadata> first = service.getSessionFiles("g1");
first.clear();
// Mutating the returned list must not affect the service's internal state.
assertEquals(1, service.getSessionFiles("g1").size());
}
}
@Nested
@DisplayName("getFile")
class GetFile {
@Test
@DisplayName("returns the path of an uploaded file")
void returnsPath() throws IOException {
service.createSession("f1");
service.uploadFiles("f1", List.of(file("doc.txt", "body")));
Path path = service.getFile("f1", "doc.txt");
assertTrue(Files.exists(path));
assertEquals("body", Files.readString(path));
}
@Test
@DisplayName("throws when the session does not exist")
void unknownSessionThrows() {
IOException ex =
assertThrows(IOException.class, () -> service.getFile("ghost", "doc.txt"));
assertTrue(ex.getMessage().contains("Session not found"));
}
@Test
@DisplayName("throws when the file does not exist in an existing session")
void unknownFileThrows() throws IOException {
service.createSession("f2");
service.uploadFiles("f2", List.of(file("present.txt", "x")));
IOException ex =
assertThrows(IOException.class, () -> service.getFile("f2", "missing.txt"));
assertTrue(ex.getMessage().contains("File not found"));
}
@Test
@DisplayName("rejects filenames containing path separators")
void rejectsPathSeparators() throws IOException {
service.createSession("f3");
service.uploadFiles("f3", List.of(file("ok.txt", "x")));
assertThrows(IOException.class, () -> service.getFile("f3", "../escape.txt"));
assertThrows(IOException.class, () -> service.getFile("f3", "sub/file.txt"));
assertThrows(IOException.class, () -> service.getFile("f3", "sub\\file.txt"));
}
@Test
@DisplayName("rejects blank filename")
void rejectsBlankFilename() throws IOException {
service.createSession("f4");
service.uploadFiles("f4", List.of(file("ok.txt", "x")));
assertThrows(IOException.class, () -> service.getFile("f4", " "));
}
}
@Nested
@DisplayName("deleteFileAfterDownload")
class DeleteFileAfterDownload {
@Test
@DisplayName("deletes a single file but keeps the session if others remain")
void deletesOneFile() throws IOException {
service.createSession("d1");
service.uploadFiles("d1", List.of(file("a.txt", "x"), file("b.txt", "y")));
service.deleteFileAfterDownload("d1", "a.txt");
assertFalse(Files.exists(tempDir.resolve("d1").resolve("a.txt")));
// Session still present because not all files have been downloaded.
assertNotNull(service.validateSession("d1"));
}
@Test
@DisplayName("deletes the entire session once all files are marked downloaded")
void deletesSessionWhenAllDownloaded() throws IOException {
service.createSession("d2");
service.uploadFiles("d2", List.of(file("only.txt", "x")));
// Mark the file as downloaded via getFile, then delete it.
service.getFile("d2", "only.txt");
service.deleteFileAfterDownload("d2", "only.txt");
assertNull(service.validateSession("d2"));
assertFalse(Files.exists(tempDir.resolve("d2")));
}
@Test
@DisplayName("does not throw for an unknown session")
void unknownSessionNoThrow() {
assertDoesNotThrow(() -> service.deleteFileAfterDownload("ghost", "a.txt"));
}
@Test
@DisplayName("swallows invalid filename input without throwing")
void invalidFilenameNoThrow() throws IOException {
service.createSession("d3");
service.uploadFiles("d3", List.of(file("a.txt", "x")));
assertDoesNotThrow(() -> service.deleteFileAfterDownload("d3", "../escape.txt"));
// Original file untouched.
assertTrue(Files.exists(tempDir.resolve("d3").resolve("a.txt")));
}
}
@Nested
@DisplayName("deleteSession")
class DeleteSession {
@Test
@DisplayName("removes the session and all its files")
void removesSessionAndFiles() throws IOException {
service.createSession("x1");
service.uploadFiles("x1", List.of(file("a.txt", "x"), file("b.txt", "y")));
assertTrue(Files.exists(tempDir.resolve("x1")));
service.deleteSession("x1");
assertNull(service.validateSession("x1"));
assertFalse(Files.exists(tempDir.resolve("x1")));
}
@Test
@DisplayName("is a no-op for an unknown session")
void unknownSessionNoOp() {
assertDoesNotThrow(() -> service.deleteSession("never-existed"));
}
}
@Nested
@DisplayName("cleanupExpiredSessions")
class CleanupExpiredSessions {
@Test
@DisplayName("removes sessions past the timeout")
void removesExpired() throws IOException {
service.createSession("old");
service.uploadFiles("old", List.of(file("a.txt", "x")));
forceLastAccess("old", System.currentTimeMillis() - (20 * 60 * 1000L));
service.cleanupExpiredSessions();
assertNull(service.validateSession("old"));
assertFalse(Files.exists(tempDir.resolve("old")));
}
@Test
@DisplayName("keeps sessions that are still fresh")
void keepsFresh() {
service.createSession("fresh");
service.cleanupExpiredSessions();
assertNotNull(service.validateSession("fresh"));
}
@Test
@DisplayName("does not throw when there are no sessions")
void noSessionsNoThrow() {
assertDoesNotThrow(() -> service.cleanupExpiredSessions());
}
}
@Nested
@DisplayName("SessionInfo accessors")
class SessionInfoAccessors {
@Test
@DisplayName("exposes all constructor values")
void exposesValues() {
SessionInfo info = new SessionInfo("id", 100L, 200L, 50L);
assertEquals("id", info.getSessionId());
assertEquals(100L, info.getCreatedAt());
assertEquals(200L, info.getExpiresAt());
assertEquals(50L, info.getTimeoutMs());
}
}
@Nested
@DisplayName("FileMetadata accessors")
class FileMetadataAccessors {
@Test
@DisplayName("exposes all constructor values")
void exposesValues() {
FileMetadata meta = new FileMetadata("name.pdf", 1234L, "application/pdf");
assertEquals("name.pdf", meta.getFilename());
assertEquals(1234L, meta.getSize());
assertEquals("application/pdf", meta.getContentType());
}
}
/**
* Reaches into the internal SessionData for a given session and forces its lastAccessTime, used
* to deterministically simulate expiry without sleeping.
*/
@SuppressWarnings("unchecked")
private void forceLastAccess(String sessionId, long lastAccessTime) {
java.util.Map<String, Object> sessions =
(java.util.Map<String, Object>)
ReflectionTestUtils.getField(service, "activeSessions");
assertNotNull(sessions);
Object sessionData = sessions.get(sessionId);
assertNotNull(sessionData, "session not found: " + sessionId);
ReflectionTestUtils.setField(sessionData, "lastAccessTime", lastAccessTime);
}
}
@@ -0,0 +1,416 @@
package stirling.software.common.service;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.mockito.Mockito.lenient;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
import java.time.LocalDateTime;
import java.time.ZoneId;
import java.time.ZonedDateTime;
import java.util.Calendar;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDDocumentInformation;
import org.apache.pdfbox.pdmodel.PDPage;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.model.ApplicationProperties.Premium;
import stirling.software.common.model.ApplicationProperties.Premium.ProFeatures;
import stirling.software.common.model.ApplicationProperties.Premium.ProFeatures.CustomMetadata;
import stirling.software.common.model.PdfMetadata;
class PdfMetadataServiceTest {
private static final String LABEL = "Stirling-PDF v1.0.0";
/**
* Builds a service whose pro-features are disabled (real ApplicationProperties, all defaults).
*/
private PdfMetadataService nonProService(UserServiceInterface userService) {
return new PdfMetadataService(new ApplicationProperties(), LABEL, false, userService);
}
@Nested
@DisplayName("toCalendar(ZonedDateTime)")
class ToCalendarTests {
@Test
@DisplayName("returns null for null input")
void nullReturnsNull() {
assertNull(PdfMetadataService.toCalendar(null));
}
@Test
@DisplayName("converts ZonedDateTime preserving the instant")
void convertsInstant() {
ZonedDateTime zdt = ZonedDateTime.of(2021, 6, 15, 10, 30, 45, 0, ZoneId.of("UTC"));
Calendar cal = PdfMetadataService.toCalendar(zdt);
assertNotNull(cal);
assertEquals(zdt.toInstant().toEpochMilli(), cal.getTimeInMillis());
}
}
@Nested
@DisplayName("parseToCalendar(String)")
class ParseToCalendarTests {
@Test
@DisplayName("returns null for null input")
void nullReturnsNull() {
assertNull(PdfMetadataService.parseToCalendar(null));
}
@Test
@DisplayName("returns null for empty / blank input")
void blankReturnsNull() {
assertNull(PdfMetadataService.parseToCalendar(""));
assertNull(PdfMetadataService.parseToCalendar(" "));
}
@Test
@DisplayName("returns null for unparsable input")
void invalidReturnsNull() {
assertNull(PdfMetadataService.parseToCalendar("not a date"));
assertNull(PdfMetadataService.parseToCalendar("2021-06-15"));
assertNull(PdfMetadataService.parseToCalendar("2021/13/40 99:99:99"));
}
@Test
@DisplayName("parses a valid 'yyyy/MM/dd HH:mm:ss' string")
void parsesValidDate() {
Calendar cal = PdfMetadataService.parseToCalendar("2021/06/15 10:30:45");
assertNotNull(cal);
// Build the expected instant the same way the implementation does so the
// assertion is independent of the JVM's default time zone.
long expectedMillis =
LocalDateTime.of(2021, 6, 15, 10, 30, 45)
.atZone(ZoneId.systemDefault())
.toInstant()
.toEpochMilli();
assertEquals(expectedMillis, cal.getTimeInMillis());
}
}
@Nested
@DisplayName("extractMetadataFromPdf(PDDocument)")
class ExtractMetadataTests {
@Test
@DisplayName("returns all-null fields for a fresh empty document")
void emptyDocumentYieldsNulls() throws Exception {
PdfMetadataService service = nonProService(null);
try (PDDocument doc = new PDDocument()) {
PdfMetadata md = service.extractMetadataFromPdf(doc);
assertNotNull(md);
assertNull(md.getAuthor());
assertNull(md.getProducer());
assertNull(md.getTitle());
assertNull(md.getCreator());
assertNull(md.getSubject());
assertNull(md.getKeywords());
assertNull(md.getCreationDate());
assertNull(md.getModificationDate());
}
}
@Test
@DisplayName("reads back string and date fields set on the document")
void readsBackPopulatedFields() throws Exception {
PdfMetadataService service = nonProService(null);
try (PDDocument doc = new PDDocument()) {
PDDocumentInformation info = doc.getDocumentInformation();
info.setAuthor("Alice");
info.setProducer("ProducerX");
info.setTitle("My Title");
info.setCreator("CreatorY");
info.setSubject("Subject Z");
info.setKeywords("k1, k2");
Calendar creation = Calendar.getInstance();
creation.setTimeInMillis(1_600_000_000_000L);
Calendar modification = Calendar.getInstance();
modification.setTimeInMillis(1_700_000_000_000L);
info.setCreationDate(creation);
info.setModificationDate(modification);
PdfMetadata md = service.extractMetadataFromPdf(doc);
assertEquals("Alice", md.getAuthor());
assertEquals("ProducerX", md.getProducer());
assertEquals("My Title", md.getTitle());
assertEquals("CreatorY", md.getCreator());
assertEquals("Subject Z", md.getSubject());
assertEquals("k1, k2", md.getKeywords());
assertNotNull(md.getCreationDate());
assertNotNull(md.getModificationDate());
assertEquals(1_600_000_000_000L, md.getCreationDate().toInstant().toEpochMilli());
assertEquals(
1_700_000_000_000L, md.getModificationDate().toInstant().toEpochMilli());
}
}
}
@Nested
@DisplayName("setMetadataToPdf / setDefaultMetadata (non-pro path)")
class SetMetadataNonProTests {
@Test
@DisplayName("writes producer label, title, subject, keywords and author from metadata")
void writesCommonMetadata() throws Exception {
PdfMetadataService service = nonProService(null);
PdfMetadata md =
PdfMetadata.builder()
.author("Bob")
.title("Doc Title")
.subject("Doc Subject")
.keywords("a, b, c")
.creationDate(
ZonedDateTime.of(2020, 1, 1, 0, 0, 0, 0, ZoneId.of("UTC")))
.modificationDate(
ZonedDateTime.of(2021, 1, 1, 0, 0, 0, 0, ZoneId.of("UTC")))
.build();
try (PDDocument doc = new PDDocument()) {
doc.addPage(new PDPage());
service.setMetadataToPdf(doc, md);
PDDocumentInformation info = doc.getDocumentInformation();
assertEquals(LABEL, info.getProducer());
assertEquals("Doc Title", info.getTitle());
assertEquals("Doc Subject", info.getSubject());
assertEquals("a, b, c", info.getKeywords());
// Non-pro: author is taken verbatim from the metadata.
assertEquals("Bob", info.getAuthor());
assertNotNull(info.getModificationDate());
}
}
@Test
@DisplayName("existing creation date is left untouched when not newly created")
void keepsExistingCreationDate() throws Exception {
PdfMetadataService service = nonProService(null);
ZonedDateTime creation = ZonedDateTime.of(2019, 5, 20, 8, 15, 0, 0, ZoneId.of("UTC"));
PdfMetadata md = PdfMetadata.builder().title("T").creationDate(creation).build();
try (PDDocument doc = new PDDocument()) {
service.setMetadataToPdf(doc, md);
Calendar creationCal = doc.getDocumentInformation().getCreationDate();
// creationDate is non-null and newlyCreated=false, so setNewDocumentMetadata
// is skipped and no creation date is written.
assertNull(creationCal);
}
}
@Test
@DisplayName("sets a fresh creation date when metadata has none")
void setsCreationDateWhenMissing() throws Exception {
PdfMetadataService service = nonProService(null);
PdfMetadata md = PdfMetadata.builder().title("T").build();
try (PDDocument doc = new PDDocument()) {
service.setMetadataToPdf(doc, md);
Calendar creationCal = doc.getDocumentInformation().getCreationDate();
assertNotNull(creationCal);
// Non-pro path writes the Stirling label as the creator.
assertEquals(LABEL, doc.getDocumentInformation().getCreator());
}
}
@Test
@DisplayName("newlyCreated=true forces a fresh creation date even if metadata has one")
void newlyCreatedForcesCreationDate() throws Exception {
PdfMetadataService service = nonProService(null);
ZonedDateTime creation = ZonedDateTime.of(2018, 3, 3, 3, 3, 3, 0, ZoneId.of("UTC"));
PdfMetadata md = PdfMetadata.builder().title("T").creationDate(creation).build();
try (PDDocument doc = new PDDocument()) {
service.setMetadataToPdf(doc, md, true);
Calendar creationCal = doc.getDocumentInformation().getCreationDate();
assertNotNull(creationCal);
// The supplied creation date must have been honoured (not "now").
assertEquals(creation.toInstant().toEpochMilli(), creationCal.getTimeInMillis());
assertEquals(LABEL, doc.getDocumentInformation().getCreator());
}
}
@Test
@DisplayName(
"setDefaultMetadata round-trips existing document info through the producer label")
void setDefaultMetadataRewritesProducer() throws Exception {
PdfMetadataService service = nonProService(null);
try (PDDocument doc = new PDDocument()) {
PDDocumentInformation info = doc.getDocumentInformation();
info.setTitle("Original Title");
info.setAuthor("Original Author");
info.setProducer("Some Other Producer");
service.setDefaultMetadata(doc);
// extract + re-apply keeps title/author but rewrites producer to the label.
assertEquals("Original Title", info.getTitle());
assertEquals("Original Author", info.getAuthor());
assertEquals(LABEL, info.getProducer());
}
}
@Test
@DisplayName("null string fields in metadata are written through without error")
void handlesNullStringFields() throws Exception {
PdfMetadataService service = nonProService(null);
PdfMetadata md = PdfMetadata.builder().build();
try (PDDocument doc = new PDDocument()) {
service.setMetadataToPdf(doc, md, true);
PDDocumentInformation info = doc.getDocumentInformation();
assertEquals(LABEL, info.getProducer());
assertNull(info.getTitle());
assertNull(info.getSubject());
assertNull(info.getKeywords());
assertNull(info.getAuthor());
// newlyCreated=true always stamps a creation date.
assertNotNull(info.getCreationDate());
assertNotNull(info.getModificationDate());
}
}
}
@Nested
@DisplayName("setMetadataToPdf (pro path with custom metadata)")
class SetMetadataProTests {
private ApplicationProperties propsWithCustomMetadata(
boolean autoUpdate, String author, String creator) {
ApplicationProperties props = mock(ApplicationProperties.class);
Premium premium = mock(Premium.class);
ProFeatures proFeatures = mock(ProFeatures.class);
CustomMetadata customMetadata = mock(CustomMetadata.class);
lenient().when(props.getPremium()).thenReturn(premium);
lenient().when(premium.getProFeatures()).thenReturn(proFeatures);
lenient().when(proFeatures.getCustomMetadata()).thenReturn(customMetadata);
lenient().when(customMetadata.isAutoUpdateMetadata()).thenReturn(autoUpdate);
lenient().when(customMetadata.getAuthor()).thenReturn(author);
lenient().when(customMetadata.getCreator()).thenReturn(creator);
return props;
}
@Test
@DisplayName("uses custom author and creator when pro and auto-update enabled")
void appliesCustomAuthorAndCreator() throws Exception {
ApplicationProperties props =
propsWithCustomMetadata(true, "Custom Author", "Custom Creator");
PdfMetadataService service = new PdfMetadataService(props, LABEL, true, null);
PdfMetadata md = PdfMetadata.builder().author("Ignored").title("T").build();
try (PDDocument doc = new PDDocument()) {
service.setMetadataToPdf(doc, md, true);
PDDocumentInformation info = doc.getDocumentInformation();
assertEquals("Custom Author", info.getAuthor());
assertEquals("Custom Creator", info.getCreator());
// Producer is set to the label by both setNewDocumentMetadata and
// setCommonMetadata.
assertEquals(LABEL, info.getProducer());
}
}
@Test
@DisplayName("replaces 'username' token with the current user when userService present")
void replacesUsernameToken() throws Exception {
ApplicationProperties props =
propsWithCustomMetadata(true, "Report by username", "Creator");
UserServiceInterface userService = mock(UserServiceInterface.class);
when(userService.getCurrentUsername()).thenReturn("alice");
PdfMetadataService service = new PdfMetadataService(props, LABEL, true, userService);
PdfMetadata md = PdfMetadata.builder().title("T").build();
try (PDDocument doc = new PDDocument()) {
service.setMetadataToPdf(doc, md, true);
assertEquals("Report by alice", doc.getDocumentInformation().getAuthor());
}
}
@Test
@DisplayName("leaves 'username' token intact when current user is null")
void keepsTokenWhenUsernameNull() throws Exception {
ApplicationProperties props =
propsWithCustomMetadata(true, "Report by username", "Creator");
UserServiceInterface userService = mock(UserServiceInterface.class);
when(userService.getCurrentUsername()).thenReturn(null);
PdfMetadataService service = new PdfMetadataService(props, LABEL, true, userService);
PdfMetadata md = PdfMetadata.builder().title("T").build();
try (PDDocument doc = new PDDocument()) {
service.setMetadataToPdf(doc, md, true);
assertEquals("Report by username", doc.getDocumentInformation().getAuthor());
}
}
@Test
@DisplayName("custom author applied even without a userService")
void appliesCustomAuthorWithoutUserService() throws Exception {
ApplicationProperties props = propsWithCustomMetadata(true, "Static Author", "Creator");
PdfMetadataService service = new PdfMetadataService(props, LABEL, true, null);
PdfMetadata md = PdfMetadata.builder().title("T").build();
try (PDDocument doc = new PDDocument()) {
service.setMetadataToPdf(doc, md, true);
assertEquals("Static Author", doc.getDocumentInformation().getAuthor());
}
}
@Test
@DisplayName("pro flag without auto-update keeps metadata author and label creator")
void proButAutoUpdateDisabledUsesMetadata() throws Exception {
ApplicationProperties props =
propsWithCustomMetadata(false, "Custom Author", "Custom Creator");
PdfMetadataService service = new PdfMetadataService(props, LABEL, true, null);
PdfMetadata md = PdfMetadata.builder().author("Metadata Author").title("T").build();
try (PDDocument doc = new PDDocument()) {
service.setMetadataToPdf(doc, md, true);
PDDocumentInformation info = doc.getDocumentInformation();
assertEquals("Metadata Author", info.getAuthor());
assertEquals(LABEL, info.getCreator());
}
}
@Test
@DisplayName("auto-update enabled but not pro keeps metadata author and label creator")
void autoUpdateButNotProUsesMetadata() throws Exception {
ApplicationProperties props =
propsWithCustomMetadata(true, "Custom Author", "Custom Creator");
PdfMetadataService service = new PdfMetadataService(props, LABEL, false, null);
PdfMetadata md = PdfMetadata.builder().author("Metadata Author").title("T").build();
try (PDDocument doc = new PDDocument()) {
service.setMetadataToPdf(doc, md, true);
PDDocumentInformation info = doc.getDocumentInformation();
assertEquals("Metadata Author", info.getAuthor());
assertEquals(LABEL, info.getCreator());
}
}
}
}
@@ -0,0 +1,441 @@
package stirling.software.common.service;
import static org.junit.jupiter.api.Assertions.*;
import static org.mockito.ArgumentMatchers.*;
import static org.mockito.Mockito.*;
import java.util.HashMap;
import java.util.Map;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.ArgumentCaptor;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import org.mockito.junit.jupiter.MockitoSettings;
import org.mockito.quality.Strictness;
import org.springframework.mock.env.MockEnvironment;
import com.posthog.java.PostHog;
import stirling.software.common.model.ApplicationProperties;
@ExtendWith(MockitoExtension.class)
@MockitoSettings(strictness = Strictness.LENIENT)
class PostHogServiceTest {
private static final String UUID = "test-uuid-1234";
private static final String APP_VERSION = "9.9.9";
@Mock PostHog postHog;
@Mock UserServiceInterface userService;
/** Build an ApplicationProperties with analytics/posthog toggled. */
private ApplicationProperties props(boolean analyticsEnabled) {
ApplicationProperties appProps = new ApplicationProperties();
appProps.getSystem().setEnableAnalytics(analyticsEnabled);
return appProps;
}
/** Construct the service under test. */
private PostHogService newService(
ApplicationProperties appProps,
UserServiceInterface user,
boolean configDirMounted,
MockEnvironment env) {
return new PostHogService(
postHog, UUID, configDirMounted, APP_VERSION, appProps, user, env);
}
private MockEnvironment env() {
return new MockEnvironment();
}
@Nested
@DisplayName("Constructor / captureSystemInfo")
class ConstructorBehavior {
@Test
@DisplayName("constructor captures system_info when posthog is enabled")
void constructorCapturesWhenEnabled() {
ApplicationProperties appProps = props(true);
newService(appProps, userService, false, env());
verify(postHog).capture(eq(UUID), eq("system_info_captured"), anyMap());
}
@Test
@DisplayName("constructor does not capture when analytics disabled")
void constructorNoCaptureWhenDisabled() {
ApplicationProperties appProps = props(false);
newService(appProps, userService, false, env());
verify(postHog, never()).capture(anyString(), anyString(), anyMap());
}
@Test
@DisplayName("constructor does not capture when posthog explicitly disabled")
void constructorNoCaptureWhenPosthogOff() {
ApplicationProperties appProps = props(true);
appProps.getSystem().setEnablePosthog(false);
newService(appProps, userService, false, env());
verify(postHog, never()).capture(anyString(), anyString(), anyMap());
}
@Test
@DisplayName("constructor swallows exceptions thrown by postHog.capture")
void constructorSwallowsCaptureException() {
ApplicationProperties appProps = props(true);
doThrow(new RuntimeException("boom"))
.when(postHog)
.capture(anyString(), anyString(), anyMap());
// Must not propagate; constructor wraps capture in try/catch.
assertDoesNotThrow(() -> newService(appProps, userService, false, env()));
}
@Test
@DisplayName("constructor works with null userService (optional dependency)")
void constructorWithNullUserService() {
ApplicationProperties appProps = props(true);
assertDoesNotThrow(() -> newService(appProps, null, false, env()));
verify(postHog).capture(eq(UUID), eq("system_info_captured"), anyMap());
}
}
@Nested
@DisplayName("captureEvent")
class CaptureEvent {
@Test
@DisplayName("captureEvent forwards to postHog when enabled and injects app_version")
void captureEventWhenEnabled() {
ApplicationProperties appProps = props(true);
PostHogService service = newService(appProps, userService, false, env());
// Reset the constructor's capture so we only assert on captureEvent.
clearInvocations(postHog);
Map<String, Object> properties = new HashMap<>();
properties.put("foo", "bar");
service.captureEvent("my_event", properties);
@SuppressWarnings("unchecked")
ArgumentCaptor<Map<String, Object>> captor = ArgumentCaptor.forClass(Map.class);
verify(postHog).capture(eq(UUID), eq("my_event"), captor.capture());
Map<String, Object> sent = captor.getValue();
assertEquals("bar", sent.get("foo"));
assertEquals(APP_VERSION, sent.get("app_version"));
}
@Test
@DisplayName("captureEvent is a no-op when analytics disabled")
void captureEventWhenDisabled() {
ApplicationProperties appProps = props(false);
PostHogService service = newService(appProps, userService, false, env());
clearInvocations(postHog);
Map<String, Object> properties = new HashMap<>();
service.captureEvent("my_event", properties);
verify(postHog, never()).capture(anyString(), anyString(), anyMap());
// app_version must not be added when disabled (early return).
assertFalse(properties.containsKey("app_version"));
}
@Test
@DisplayName("captureEvent adds app_version key to the provided map")
void captureEventMutatesMap() {
ApplicationProperties appProps = props(true);
PostHogService service = newService(appProps, userService, false, env());
clearInvocations(postHog);
Map<String, Object> properties = new HashMap<>();
service.captureEvent("evt", properties);
assertTrue(properties.containsKey("app_version"));
assertEquals(APP_VERSION, properties.get("app_version"));
}
}
@Nested
@DisplayName("captureServerMetrics")
class CaptureServerMetrics {
private PostHogService disabledService() {
// Keep posthog disabled so the constructor performs no capture; metrics
// methods are independent of the enabled flag.
return newService(props(false), userService, true, env());
}
@Test
@DisplayName("includes core application and system metrics")
void includesCoreMetrics() {
PostHogService service = disabledService();
Map<String, Object> metrics = service.captureServerMetrics();
assertEquals(APP_VERSION, metrics.get("app_version"));
assertEquals(true, metrics.get("mounted_config_dir"));
assertNotNull(metrics.get("os_name"));
assertNotNull(metrics.get("java_version"));
assertTrue(metrics.containsKey("cpu_cores"));
assertTrue(metrics.containsKey("total_memory"));
assertTrue(metrics.containsKey("free_memory"));
assertTrue(metrics.containsKey("process_id"));
assertTrue(metrics.containsKey("jvm_uptime_ms"));
assertTrue(metrics.containsKey("thread_count"));
}
@Test
@DisplayName("deployment_type defaults to JAR when not docker/exe")
void deploymentTypeJar() {
PostHogService service = disabledService();
Map<String, Object> metrics = service.captureServerMetrics();
// In the unit-test environment there is no /.dockerenv and no BROWSER_OPEN.
assertEquals("JAR", metrics.get("deployment_type"));
}
@Test
@DisplayName("deployment_type becomes EXE when BROWSER_OPEN=true")
void deploymentTypeExe() {
MockEnvironment environment = env();
environment.setProperty("BROWSER_OPEN", "true");
PostHogService service = newService(props(false), userService, false, environment);
Map<String, Object> metrics = service.captureServerMetrics();
assertEquals("EXE", metrics.get("deployment_type"));
}
@Test
@DisplayName("BROWSER_OPEN matching is case-insensitive")
void deploymentTypeExeCaseInsensitive() {
MockEnvironment environment = env();
environment.setProperty("BROWSER_OPEN", "TRUE");
PostHogService service = newService(props(false), userService, false, environment);
Map<String, Object> metrics = service.captureServerMetrics();
assertEquals("EXE", metrics.get("deployment_type"));
}
@Test
@DisplayName("mounted_config_dir reflects the configDirMounted flag")
void mountedConfigDirFalse() {
PostHogService service = newService(props(false), userService, false, env());
Map<String, Object> metrics = service.captureServerMetrics();
assertEquals(false, metrics.get("mounted_config_dir"));
}
@Test
@DisplayName("includes total_users_created when userService present")
void includesUserCountWhenUserServicePresent() {
when(userService.getTotalUsersCount()).thenReturn(42L);
PostHogService service = newService(props(false), userService, false, env());
Map<String, Object> metrics = service.captureServerMetrics();
assertEquals(42L, metrics.get("total_users_created"));
}
@Test
@DisplayName("omits total_users_created when userService is null")
void omitsUserCountWhenUserServiceNull() {
PostHogService service = newService(props(false), null, false, env());
Map<String, Object> metrics = service.captureServerMetrics();
assertFalse(metrics.containsKey("total_users_created"));
}
@Test
@DisplayName("always embeds nested application_properties map")
void embedsApplicationProperties() {
PostHogService service = disabledService();
Map<String, Object> metrics = service.captureServerMetrics();
assertTrue(metrics.get("application_properties") instanceof Map);
}
}
@Nested
@DisplayName("captureApplicationProperties")
class CaptureApplicationProperties {
private PostHogService serviceWith(ApplicationProperties appProps) {
// Disable analytics to keep the constructor from capturing.
appProps.getSystem().setEnableAnalytics(false);
return newService(appProps, userService, false, env());
}
@Test
@DisplayName("includes blank-trimmed legal strings only when non-empty")
void legalPropertiesFiltered() {
ApplicationProperties appProps = new ApplicationProperties();
appProps.getLegal().setTermsAndConditions(" https://terms ");
appProps.getLegal().setPrivacyPolicy(""); // blank -> skipped
PostHogService service = serviceWith(appProps);
Map<String, Object> p = service.captureApplicationProperties();
// String values are trimmed by addIfNotEmpty.
assertEquals("https://terms", p.get("legal_termsAndConditions"));
assertFalse(p.containsKey("legal_privacyPolicy"));
assertFalse(p.containsKey("legal_accessibilityStatement"));
}
@Test
@DisplayName("always reports csrfDisabled true and login booleans")
void securityProperties() {
ApplicationProperties appProps = new ApplicationProperties();
appProps.getSecurity().setEnableLogin(true);
appProps.getSecurity().setLoginAttemptCount(5);
appProps.getSecurity().setLoginResetTimeMinutes(10);
PostHogService service = serviceWith(appProps);
Map<String, Object> p = service.captureApplicationProperties();
assertEquals(true, p.get("security_csrfDisabled"));
assertEquals(true, p.get("security_enableLogin"));
assertEquals(5, p.get("security_loginAttemptCount"));
assertEquals(10L, p.get("security_loginResetTimeMinutes"));
assertEquals("all", p.get("security_loginMethod"));
}
@Test
@DisplayName("oauth2 nested fields are omitted when oauth2 disabled")
void oauth2DisabledOmitsNested() {
ApplicationProperties appProps = new ApplicationProperties();
// oauth2.enabled defaults to false.
PostHogService service = serviceWith(appProps);
Map<String, Object> p = service.captureApplicationProperties();
assertEquals(false, p.get("security_oauth2_enabled"));
assertFalse(p.containsKey("security_oauth2_autoCreateUser"));
assertFalse(p.containsKey("security_oauth2_provider"));
}
@Test
@DisplayName("oauth2 nested fields are included when oauth2 enabled")
void oauth2EnabledIncludesNested() {
ApplicationProperties appProps = new ApplicationProperties();
appProps.getSecurity().getOauth2().setEnabled(true);
appProps.getSecurity().getOauth2().setAutoCreateUser(true);
appProps.getSecurity().getOauth2().setBlockRegistration(false);
appProps.getSecurity().getOauth2().setUseAsUsername("email");
appProps.getSecurity().getOauth2().setProvider("google");
PostHogService service = serviceWith(appProps);
Map<String, Object> p = service.captureApplicationProperties();
assertEquals(true, p.get("security_oauth2_enabled"));
assertEquals(true, p.get("security_oauth2_autoCreateUser"));
assertEquals(false, p.get("security_oauth2_blockRegistration"));
assertEquals("email", p.get("security_oauth2_useAsUsername"));
assertEquals("google", p.get("security_oauth2_provider"));
}
@Test
@DisplayName("system analytics/posthog/scarf booleans are reported")
void systemAnalyticsBooleans() {
ApplicationProperties appProps = new ApplicationProperties();
appProps.getSystem().setEnableAnalytics(true);
appProps.getSystem().setEnablePosthog(true);
appProps.getSystem().setEnableScarf(false);
appProps.getSystem().setDefaultLocale("en-US");
PostHogService service = newService(appProps, userService, false, env());
// Constructor will capture once because analytics is enabled; that's fine.
clearInvocations(postHog);
Map<String, Object> p = service.captureApplicationProperties();
assertEquals("en-US", p.get("system_defaultLocale"));
assertEquals(true, p.get("system_enableAnalytics"));
assertEquals(true, p.get("system_enablePosthog"));
// isScarfEnabled() is false because enableScarf is false.
assertEquals(false, p.get("system_enableScarf"));
}
@Test
@DisplayName("metrics_enabled and autoPipeline output folder included appropriately")
void metricsAndAutoPipeline() {
ApplicationProperties appProps = new ApplicationProperties();
appProps.getMetrics().setEnabled(true);
appProps.getAutoPipeline().setOutputFolder("/tmp/out");
PostHogService service = serviceWith(appProps);
Map<String, Object> p = service.captureApplicationProperties();
assertEquals(true, p.get("metrics_enabled"));
assertEquals("/tmp/out", p.get("autoPipeline_outputFolder"));
}
@Test
@DisplayName("enterprise metadata flag omitted when premium disabled")
void premiumDisabledOmitsMetadata() {
ApplicationProperties appProps = new ApplicationProperties();
// premium.enabled defaults to false.
PostHogService service = serviceWith(appProps);
Map<String, Object> p = service.captureApplicationProperties();
assertEquals(false, p.get("enterpriseEdition_enabled"));
assertFalse(p.containsKey("enterpriseEdition_customMetadata_autoUpdateMetadata"));
}
@Test
@DisplayName("enterprise metadata flag included when premium enabled")
void premiumEnabledIncludesMetadata() {
ApplicationProperties appProps = new ApplicationProperties();
appProps.getPremium().setEnabled(true);
appProps.getPremium().getProFeatures().getCustomMetadata().setAutoUpdateMetadata(true);
PostHogService service = serviceWith(appProps);
Map<String, Object> p = service.captureApplicationProperties();
assertEquals(true, p.get("enterpriseEdition_enabled"));
assertEquals(true, p.get("enterpriseEdition_customMetadata_autoUpdateMetadata"));
}
@Test
@DisplayName("ui appNameNavbar omitted when blank, included when set")
void uiAppNameNavbar() {
ApplicationProperties blankProps = new ApplicationProperties();
// appNameNavbar getter returns null for blank/empty values.
PostHogService blankService = serviceWith(blankProps);
Map<String, Object> blank = blankService.captureApplicationProperties();
assertFalse(blank.containsKey("ui_appNameNavbar"));
ApplicationProperties namedProps = new ApplicationProperties();
namedProps.getUi().setAppNameNavbar("My App");
PostHogService namedService = serviceWith(namedProps);
Map<String, Object> named = namedService.captureApplicationProperties();
assertEquals("My App", named.get("ui_appNameNavbar"));
}
@Test
@DisplayName("returns a non-null map for a fresh ApplicationProperties")
void defaultsProduceNonNullMap() {
PostHogService service = serviceWith(new ApplicationProperties());
Map<String, Object> p = service.captureApplicationProperties();
assertNotNull(p);
// csrfDisabled is always added regardless of config, so map is never empty.
assertTrue(p.containsKey("security_csrfDisabled"));
}
}
}
@@ -0,0 +1,738 @@
package stirling.software.common.util;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertInstanceOf;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertSame;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.mockito.Mockito.mockStatic;
import java.io.IOException;
import java.util.List;
import org.apache.pdfbox.pdmodel.PDPage;
import org.apache.pdfbox.pdmodel.common.PDRectangle;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test;
import org.mockito.MockedStatic;
import stirling.software.common.util.ExceptionUtils.BaseAppException;
import stirling.software.common.util.ExceptionUtils.CbrFormatException;
import stirling.software.common.util.ExceptionUtils.CbzFormatException;
import stirling.software.common.util.ExceptionUtils.EmlFormatException;
import stirling.software.common.util.ExceptionUtils.ErrorCode;
import stirling.software.common.util.ExceptionUtils.FfmpegRequiredException;
import stirling.software.common.util.ExceptionUtils.GhostscriptException;
import stirling.software.common.util.ExceptionUtils.OutOfMemoryDpiException;
import stirling.software.common.util.ExceptionUtils.PdfCorruptedException;
/**
* Additional gap-filling unit tests for {@link ExceptionUtils}, covering areas not exercised by
* {@code ExceptionUtilsTest}: CBR/CBZ/EML factories, error-code hint/action lookups, rendering
* dimension validation, OOM rendering wrappers, Ghostscript output analysis, and wrapException.
*
* <p>The {@code messages} ResourceBundle is not on the common module test classpath, so {@link
* ExceptionUtils} falls back to the default messages baked into {@link ErrorCode}. Assertions here
* rely only on those default messages and on deterministic structural behavior.
*/
class ExceptionUtilsGapTest {
@Nested
@DisplayName("ErrorCode enum metadata")
class ErrorCodeMetadataTests {
@Test
@DisplayName("each error code exposes code, message key and default message")
void allErrorCodesHaveMetadata() {
for (ErrorCode code : ErrorCode.values()) {
assertNotNull(code.getCode(), "code for " + code);
assertTrue(code.getCode().startsWith("E"), "code prefix for " + code);
assertNotNull(code.getMessageKey(), "messageKey for " + code);
assertNotNull(code.getDefaultMessage(), "defaultMessage for " + code);
assertFalse(code.getDefaultMessage().isEmpty(), "defaultMessage empty for " + code);
}
}
@Test
@DisplayName("known error codes map to expected identifiers")
void knownErrorCodeIdentifiers() {
assertEquals("E001", ErrorCode.PDF_CORRUPTED.getCode());
assertEquals("E081", ErrorCode.OUT_OF_MEMORY_DPI.getCode());
assertEquals("error.pdfCorrupted", ErrorCode.PDF_CORRUPTED.getMessageKey());
}
}
@Nested
@DisplayName("Hints and action lookups via resource bundle")
class HintAndActionTests {
@Test
@DisplayName("getHintsForErrorCode returns empty list for null code")
void hintsNullCode() {
assertEquals(List.of(), ExceptionUtils.getHintsForErrorCode(null));
}
@Test
@DisplayName("getHintsForErrorCode returns empty list when no hints exist in bundle")
void hintsMissingFromBundle() {
// Fallback empty bundle has no hint keys, so the result is an empty list.
List<String> hints = ExceptionUtils.getHintsForErrorCode("E001");
assertNotNull(hints);
assertTrue(hints.isEmpty());
}
@Test
@DisplayName("getActionRequiredForErrorCode returns null for null code")
void actionNullCode() {
assertNull(ExceptionUtils.getActionRequiredForErrorCode(null));
}
@Test
@DisplayName("getActionRequiredForErrorCode returns null when key absent from bundle")
void actionMissingFromBundle() {
assertNull(ExceptionUtils.getActionRequiredForErrorCode("E001"));
}
}
@Nested
@DisplayName("CBR format exception factories")
class CbrFactoryTests {
@Test
@DisplayName("invalid format uses provided message when non-null")
void cbrInvalidFormatWithMessage() {
CbrFormatException ex =
ExceptionUtils.createCbrInvalidFormatException("custom cbr msg");
assertEquals("custom cbr msg", ex.getMessage());
assertEquals(ErrorCode.CBR_INVALID_FORMAT.getCode(), ex.getErrorCode());
}
@Test
@DisplayName("invalid format falls back to default message when null")
void cbrInvalidFormatNullMessage() {
CbrFormatException ex = ExceptionUtils.createCbrInvalidFormatException(null);
assertTrue(ex.getMessage().contains("CBR/RAR archive"));
assertEquals("E010", ex.getErrorCode());
}
@Test
@DisplayName("encrypted CBR reuses invalid-format code")
void cbrEncrypted() {
CbrFormatException ex = ExceptionUtils.createCbrEncryptedException();
assertEquals(ErrorCode.CBR_INVALID_FORMAT.getCode(), ex.getErrorCode());
}
@Test
@DisplayName("no images and corrupted images both map to CBR_NO_IMAGES")
void cbrNoImages() {
CbrFormatException noImages = ExceptionUtils.createCbrNoImagesException();
CbrFormatException corrupted = ExceptionUtils.createCbrCorruptedImagesException();
assertEquals(ErrorCode.CBR_NO_IMAGES.getCode(), noImages.getErrorCode());
assertEquals(ErrorCode.CBR_NO_IMAGES.getCode(), corrupted.getErrorCode());
assertTrue(noImages.getMessage().contains("No valid images"));
}
@Test
@DisplayName("not-a-CBR file uses CBR_NOT_CBR code")
void notCbr() {
CbrFormatException ex = ExceptionUtils.createNotCbrFileException();
assertEquals(ErrorCode.CBR_NOT_CBR.getCode(), ex.getErrorCode());
assertTrue(ex.getMessage().contains("CBR or RAR"));
}
@Test
@DisplayName(
"CbrFormatException is an IllegalArgumentException via BaseValidationException")
void cbrIsIllegalArgument() {
CbrFormatException ex = ExceptionUtils.createNotCbrFileException();
assertInstanceOf(IllegalArgumentException.class, ex);
}
}
@Nested
@DisplayName("CBZ format exception factories")
class CbzFactoryTests {
@Test
@DisplayName("invalid format wraps cause and uses CBZ_INVALID_FORMAT code")
void cbzInvalidFormat() {
Exception cause = new Exception("zip boom");
CbzFormatException ex = ExceptionUtils.createCbzInvalidFormatException(cause);
assertSame(cause, ex.getCause());
assertEquals(ErrorCode.CBZ_INVALID_FORMAT.getCode(), ex.getErrorCode());
assertTrue(ex.getMessage().contains("CBZ/ZIP archive"));
}
@Test
@DisplayName("empty CBZ reuses invalid-format code")
void cbzEmpty() {
CbzFormatException ex = ExceptionUtils.createCbzEmptyException();
assertEquals(ErrorCode.CBZ_INVALID_FORMAT.getCode(), ex.getErrorCode());
}
@Test
@DisplayName("no images and corrupted images both map to CBZ_NO_IMAGES")
void cbzNoImages() {
CbzFormatException noImages = ExceptionUtils.createCbzNoImagesException();
CbzFormatException corrupted = ExceptionUtils.createCbzCorruptedImagesException();
assertEquals(ErrorCode.CBZ_NO_IMAGES.getCode(), noImages.getErrorCode());
assertEquals(ErrorCode.CBZ_NO_IMAGES.getCode(), corrupted.getErrorCode());
}
@Test
@DisplayName("not-a-CBZ file uses CBZ_NOT_CBZ code")
void notCbz() {
CbzFormatException ex = ExceptionUtils.createNotCbzFileException();
assertEquals(ErrorCode.CBZ_NOT_CBZ.getCode(), ex.getErrorCode());
assertTrue(ex.getMessage().contains("CBZ or ZIP"));
}
}
@Nested
@DisplayName("EML format exception factories")
class EmlFactoryTests {
@Test
@DisplayName("empty EML uses EML_EMPTY code")
void emlEmpty() {
EmlFormatException ex = ExceptionUtils.createEmlEmptyException();
assertEquals(ErrorCode.EML_EMPTY.getCode(), ex.getErrorCode());
assertTrue(ex.getMessage().contains("EML file is empty"));
}
@Test
@DisplayName("invalid EML uses EML_INVALID_FORMAT code")
void emlInvalid() {
EmlFormatException ex = ExceptionUtils.createEmlInvalidFormatException();
assertEquals(ErrorCode.EML_INVALID_FORMAT.getCode(), ex.getErrorCode());
assertTrue(ex.getMessage().contains("Invalid EML"));
}
}
@Nested
@DisplayName("Image, OCR and processing factories")
class ImageOcrProcessingTests {
@Test
@DisplayName("image read exception embeds filename and has no cause")
void imageRead() {
IOException ex = ExceptionUtils.createImageReadException("photo.png");
assertTrue(ex.getMessage().contains("photo.png"));
assertNull(ex.getCause());
}
@Test
@DisplayName("image read exception rejects null filename")
void imageReadNullFilename() {
assertThrows(
IllegalArgumentException.class,
() -> ExceptionUtils.createImageReadException(null));
}
@Test
@DisplayName("ocr invalid render type uses default message")
void ocrInvalidRenderType() {
IOException ex = ExceptionUtils.createOcrInvalidRenderTypeException();
assertTrue(ex.getMessage().contains("hocr"));
}
@Test
@DisplayName("ocr processing failed includes return code")
void ocrProcessingFailed() {
IOException ex = ExceptionUtils.createOcrProcessingFailedException(7);
assertTrue(ex.getMessage().contains("7"));
}
@Test
@DisplayName("processing interrupted wraps the InterruptedException cause")
void processingInterrupted() {
InterruptedException cause = new InterruptedException("stop");
IOException ex =
ExceptionUtils.createProcessingInterruptedException("compression", cause);
assertSame(cause, ex.getCause());
assertTrue(ex.getMessage().contains("compression"));
}
@Test
@DisplayName("processing interrupted rejects null arguments")
void processingInterruptedNullArgs() {
assertThrows(
IllegalArgumentException.class,
() ->
ExceptionUtils.createProcessingInterruptedException(
null, new InterruptedException()));
assertThrows(
IllegalArgumentException.class,
() -> ExceptionUtils.createProcessingInterruptedException("x", null));
}
@Test
@DisplayName("ghostscript conversion exception embeds output type")
void ghostscriptConversion() {
IOException ex = ExceptionUtils.createGhostscriptConversionException("png");
assertNotNull(ex.getMessage());
assertThrows(
IllegalArgumentException.class,
() -> ExceptionUtils.createGhostscriptConversionException(null));
}
}
@Nested
@DisplayName("Validation factories: page size, file, ffmpeg")
class ValidationFactoryTests {
@Test
@DisplayName("invalid page size rejects null size")
void invalidPageSizeNull() {
assertThrows(
IllegalArgumentException.class,
() -> ExceptionUtils.createInvalidPageSizeException(null));
}
@Test
@DisplayName("file null-or-empty uses FILE_NULL_OR_EMPTY default message")
void fileNullOrEmpty() {
IllegalArgumentException ex = ExceptionUtils.createFileNullOrEmptyException();
assertTrue(ex.getMessage().contains("null or empty"));
}
@Test
@DisplayName("file no-name uses FILE_NO_NAME default message")
void fileNoName() {
IllegalArgumentException ex = ExceptionUtils.createFileNoNameException();
assertTrue(ex.getMessage().contains("must have a name"));
}
@Test
@DisplayName("pdf no-pages uses PDF_NO_PAGES default message")
void pdfNoPages() {
IllegalArgumentException ex = ExceptionUtils.createPdfNoPages();
assertTrue(ex.getMessage().contains("no pages"));
}
@Test
@DisplayName("ffmpeg required exception exposes FFMPEG_REQUIRED code and null cause")
void ffmpegRequired() {
FfmpegRequiredException ex = ExceptionUtils.createFfmpegRequiredException();
assertEquals(ErrorCode.FFMPEG_REQUIRED.getCode(), ex.getErrorCode());
assertNull(ex.getCause());
assertTrue(ex.getMessage().contains("FFmpeg"));
}
}
@Nested
@DisplayName("ErrorCode-based argument and IO factories")
class ErrorCodeArgFactoryTests {
@Test
@DisplayName("createIllegalArgumentException(ErrorCode, args) formats default message")
void illegalArgumentFromErrorCode() {
IllegalArgumentException ex =
ExceptionUtils.createIllegalArgumentException(
ErrorCode.INVALID_PAGE_SIZE, "B7");
assertTrue(ex.getMessage().contains("B7"));
}
@Test
@DisplayName("createIllegalArgumentException rejects null ErrorCode")
void illegalArgumentFromNullErrorCode() {
ErrorCode nullCode = null;
assertThrows(
IllegalArgumentException.class,
() -> ExceptionUtils.createIllegalArgumentException(nullCode));
}
@Test
@DisplayName("createFileProcessingException rejects null operation and cause")
void fileProcessingNullArgs() {
assertThrows(
IllegalArgumentException.class,
() -> ExceptionUtils.createFileProcessingException(null, new Exception()));
assertThrows(
IllegalArgumentException.class,
() -> ExceptionUtils.createFileProcessingException("op", null));
}
@Test
@DisplayName("createInvalidArgumentException rejects null name or value")
void invalidArgumentNullArgs() {
assertThrows(
IllegalArgumentException.class,
() -> ExceptionUtils.createInvalidArgumentException(null, "v"));
assertThrows(
IllegalArgumentException.class,
() -> ExceptionUtils.createInvalidArgumentException("n", null));
}
@Test
@DisplayName("createNullArgumentException rejects null argument name")
void nullArgumentNullName() {
assertThrows(
IllegalArgumentException.class,
() -> ExceptionUtils.createNullArgumentException(null));
}
@Test
@DisplayName("createIOException without cause leaves cause null")
void ioExceptionWithoutCause() {
IOException ex = ExceptionUtils.createIOException("key", "msg {0}", null, "A");
assertEquals("msg A", ex.getMessage());
assertNull(ex.getCause());
}
@Test
@DisplayName("createRuntimeException without cause leaves cause null")
void runtimeExceptionWithoutCause() {
RuntimeException ex =
ExceptionUtils.createRuntimeException("key", "msg {0}", null, "B");
assertEquals("msg B", ex.getMessage());
assertNull(ex.getCause());
}
}
@Nested
@DisplayName("createPdfCorruptedException null-cause handling")
class PdfCorruptedCauseTests {
@Test
@DisplayName("rejects null cause")
void nullCause() {
assertThrows(
IllegalArgumentException.class,
() -> ExceptionUtils.createPdfCorruptedException("ctx", null));
}
@Test
@DisplayName("empty context behaves like no context")
void emptyContext() {
PdfCorruptedException ex =
ExceptionUtils.createPdfCorruptedException("", new Exception("x"));
assertTrue(ex.getMessage().contains("PDF file appears to be corrupted"));
assertEquals(ErrorCode.PDF_CORRUPTED.getCode(), ex.getErrorCode());
}
}
@Nested
@DisplayName("validateRenderingDimensions")
class ValidateRenderingDimensionsTests {
@Test
@DisplayName("null page is a no-op")
void nullPage() {
// Should simply return without throwing.
org.junit.jupiter.api.Assertions.assertDoesNotThrow(
() -> ExceptionUtils.validateRenderingDimensions(null, 1, 300));
}
@Test
@DisplayName("normal letter-size page at 150 DPI passes validation")
void normalPagePasses() {
PDPage page = new PDPage(PDRectangle.LETTER);
org.junit.jupiter.api.Assertions.assertDoesNotThrow(
() -> ExceptionUtils.validateRenderingDimensions(page, 1, 150));
}
@Test
@DisplayName("page with zero DPI yields zero pixels and passes")
void zeroDpiPasses() {
PDPage page = new PDPage(PDRectangle.A4);
org.junit.jupiter.api.Assertions.assertDoesNotThrow(
() -> ExceptionUtils.validateRenderingDimensions(page, 2, 0));
}
}
@Nested
@DisplayName("handleOomRendering wrappers")
class HandleOomRenderingTests {
@Test
@DisplayName("returns operation result on success (with page number)")
void successWithPage() throws IOException {
String result = ExceptionUtils.handleOomRendering(3, 300, () -> "ok");
assertEquals("ok", result);
}
@Test
@DisplayName("returns operation result on success (no page number)")
void successNoPage() throws IOException {
String result = ExceptionUtils.handleOomRendering(300, () -> "fine");
assertEquals("fine", result);
}
@Test
@DisplayName("propagates IOException from the operation unchanged")
void propagatesIoException() {
IOException boom = new IOException("io boom");
IOException thrown =
assertThrows(
IOException.class,
() ->
ExceptionUtils.handleOomRendering(
1,
300,
() -> {
throw boom;
}));
assertSame(boom, thrown);
}
@Test
@DisplayName("converts OutOfMemoryError to OutOfMemoryDpiException (with page)")
void oomToDpiExceptionWithPage() {
OutOfMemoryDpiException thrown =
assertThrows(
OutOfMemoryDpiException.class,
() ->
ExceptionUtils.handleOomRendering(
5,
300,
() -> {
throw new OutOfMemoryError("heap");
}));
assertEquals(ErrorCode.OUT_OF_MEMORY_DPI.getCode(), thrown.getErrorCode());
assertInstanceOf(OutOfMemoryError.class, thrown.getCause());
}
@Test
@DisplayName("converts NegativeArraySizeException to OutOfMemoryDpiException (no page)")
void negativeArraySizeToDpiExceptionNoPage() {
OutOfMemoryDpiException thrown =
assertThrows(
OutOfMemoryDpiException.class,
() ->
ExceptionUtils.handleOomRendering(
300,
() -> {
throw new NegativeArraySizeException("-1");
}));
assertEquals(ErrorCode.OUT_OF_MEMORY_DPI.getCode(), thrown.getErrorCode());
assertInstanceOf(NegativeArraySizeException.class, thrown.getCause());
}
}
@Nested
@DisplayName("createOutOfMemoryDpiException overloads")
class OutOfMemoryDpiFactoryTests {
@Test
@DisplayName("page + dpi + Throwable wraps cause and sets code")
void pageDpiThrowable() {
Throwable cause = new IllegalStateException("too big");
OutOfMemoryDpiException ex =
ExceptionUtils.createOutOfMemoryDpiException(4, 600, cause);
assertSame(cause, ex.getCause());
assertEquals(ErrorCode.OUT_OF_MEMORY_DPI.getCode(), ex.getErrorCode());
}
@Test
@DisplayName("page + dpi + OutOfMemoryError overload wraps the error")
void pageDpiOomError() {
OutOfMemoryError cause = new OutOfMemoryError("oom");
OutOfMemoryDpiException ex =
ExceptionUtils.createOutOfMemoryDpiException(2, 300, cause);
assertSame(cause, ex.getCause());
}
@Test
@DisplayName("dpi + Throwable overload wraps cause")
void dpiThrowable() {
Throwable cause = new RuntimeException("x");
OutOfMemoryDpiException ex = ExceptionUtils.createOutOfMemoryDpiException(300, cause);
assertSame(cause, ex.getCause());
assertEquals(ErrorCode.OUT_OF_MEMORY_DPI.getCode(), ex.getErrorCode());
}
@Test
@DisplayName("dpi + OutOfMemoryError overload wraps the error")
void dpiOomError() {
OutOfMemoryError cause = new OutOfMemoryError("oom");
OutOfMemoryDpiException ex = ExceptionUtils.createOutOfMemoryDpiException(300, cause);
assertSame(cause, ex.getCause());
}
@Test
@DisplayName("rejects null cause")
void nullCause() {
assertThrows(
IllegalArgumentException.class,
() -> ExceptionUtils.createOutOfMemoryDpiException(1, 300, (Throwable) null));
}
}
@Nested
@DisplayName("Ghostscript output analysis")
class GhostscriptAnalysisTests {
@Test
@DisplayName("null/blank output produces generic compression exception")
void blankOutput() {
GhostscriptException ex = ExceptionUtils.createGhostscriptCompressionException(" ");
assertEquals(ErrorCode.GHOSTSCRIPT_COMPRESSION.getCode(), ex.getErrorCode());
}
@Test
@DisplayName("recognized page drawing error yields page-drawing error code")
void pageDrawingError() {
String output = "Page 3\nERROR: page drawing error encountered while processing";
GhostscriptException ex = ExceptionUtils.createGhostscriptCompressionException(output);
assertEquals(ErrorCode.GHOSTSCRIPT_PAGE_DRAWING.getCode(), ex.getErrorCode());
}
@Test
@DisplayName("non-page-drawing output falls back to compression error code")
void unrecognizedOutput() {
String output = "Some random ghostscript chatter that is not an error marker";
GhostscriptException ex = ExceptionUtils.createGhostscriptCompressionException(output);
assertEquals(ErrorCode.GHOSTSCRIPT_COMPRESSION.getCode(), ex.getErrorCode());
}
@Test
@DisplayName("detectGhostscriptCriticalError returns exception only for critical output")
void detectCritical() {
GhostscriptException critical =
ExceptionUtils.detectGhostscriptCriticalError(
"Page 1\ncould not draw this page");
assertNotNull(critical);
assertEquals(ErrorCode.GHOSTSCRIPT_PAGE_DRAWING.getCode(), critical.getErrorCode());
}
@Test
@DisplayName("detectGhostscriptCriticalError returns null for non-critical output")
void detectNonCritical() {
assertNull(ExceptionUtils.detectGhostscriptCriticalError("just informational output"));
assertNull(ExceptionUtils.detectGhostscriptCriticalError(null));
}
@Test
@DisplayName("compression exception derived from cause message")
void compressionFromCauseMessage() {
GhostscriptException ex =
ExceptionUtils.createGhostscriptCompressionException(
new Exception("Page 2\npage drawing error"));
assertEquals(ErrorCode.GHOSTSCRIPT_PAGE_DRAWING.getCode(), ex.getErrorCode());
}
@Test
@DisplayName("createGhostscriptCompressionException rejects null cause overload")
void compressionNullCause() {
assertThrows(
IllegalArgumentException.class,
() -> ExceptionUtils.createGhostscriptCompressionException((Exception) null));
}
@Test
@DisplayName("multiple affected pages are summarized in the message")
void multiplePages() {
String output = "Page 1\npage drawing error\nPage 2\ncould not draw this page";
GhostscriptException ex = ExceptionUtils.createGhostscriptCompressionException(output);
assertEquals(ErrorCode.GHOSTSCRIPT_PAGE_DRAWING.getCode(), ex.getErrorCode());
assertNotNull(ex.getMessage());
}
}
@Nested
@DisplayName("wrapException")
class WrapExceptionTests {
@Test
@DisplayName("RuntimeException is returned unchanged")
void runtimePassthrough() {
RuntimeException original = new IllegalStateException("boom");
RuntimeException wrapped = ExceptionUtils.wrapException(original, "merge");
assertSame(original, wrapped);
}
@Test
@DisplayName("BaseAppException (IOException subtype) is wrapped in a RuntimeException")
void baseAppExceptionWrapped() {
// A corrupted-pdf IOException triggers handlePdfException -> PdfCorruptedException.
IOException corrupted = new IOException("Invalid PDF");
try (MockedStatic<PdfErrorUtils> mock = mockStatic(PdfErrorUtils.class)) {
mock.when(() -> PdfErrorUtils.isCorruptedPdfError(corrupted)).thenReturn(true);
RuntimeException wrapped = ExceptionUtils.wrapException(corrupted, "merge");
assertInstanceOf(BaseAppException.class, wrapped.getCause());
}
}
@Test
@DisplayName("plain IOException is wrapped via file-processing exception")
void plainIoExceptionWrapped() {
IOException io = new IOException("disk full");
try (MockedStatic<PdfErrorUtils> mock = mockStatic(PdfErrorUtils.class)) {
mock.when(() -> PdfErrorUtils.isCorruptedPdfError(io)).thenReturn(false);
RuntimeException wrapped = ExceptionUtils.wrapException(io, "split");
assertInstanceOf(IOException.class, wrapped.getCause());
assertFalse(wrapped.getCause() instanceof BaseAppException);
}
}
@Test
@DisplayName("checked non-IO exception is wrapped with operation context")
void checkedExceptionWrapped() {
Exception checked = new Exception("oops");
RuntimeException wrapped = ExceptionUtils.wrapException(checked, "convert");
assertSame(checked, wrapped.getCause());
assertTrue(wrapped.getMessage().contains("convert"));
assertTrue(wrapped.getMessage().contains("oops"));
}
@Test
@DisplayName("rejects null exception or operation")
void wrapNullArgs() {
assertThrows(
IllegalArgumentException.class, () -> ExceptionUtils.wrapException(null, "op"));
assertThrows(
IllegalArgumentException.class,
() -> ExceptionUtils.wrapException(new Exception(), null));
}
}
@Nested
@DisplayName("logException return value and handlePdfException null guard")
class LogAndHandleTests {
@Test
@DisplayName("logException returns the same exception instance for fluent throw")
void logExceptionReturnsSame() {
Exception e = new RuntimeException("unexpected");
try (MockedStatic<PdfErrorUtils> mock = mockStatic(PdfErrorUtils.class)) {
mock.when(() -> PdfErrorUtils.isCorruptedPdfError(e)).thenReturn(false);
Exception returned = ExceptionUtils.logException("op", e);
assertSame(e, returned);
}
}
@Test
@DisplayName("logException rejects null operation or exception")
void logExceptionNullArgs() {
assertThrows(
IllegalArgumentException.class,
() -> ExceptionUtils.logException(null, new Exception()));
assertThrows(
IllegalArgumentException.class, () -> ExceptionUtils.logException("op", null));
}
@Test
@DisplayName("handlePdfException rejects null exception")
void handlePdfNull() {
assertThrows(
IllegalArgumentException.class, () -> ExceptionUtils.handlePdfException(null));
}
@Test
@DisplayName("handlePdfException with context wraps corrupted PDF and includes context")
void handlePdfWithContext() {
IOException original = new IOException("damaged");
try (MockedStatic<PdfErrorUtils> mock = mockStatic(PdfErrorUtils.class)) {
mock.when(() -> PdfErrorUtils.isCorruptedPdfError(original)).thenReturn(true);
IOException result = ExceptionUtils.handlePdfException(original, "during merge");
assertInstanceOf(PdfCorruptedException.class, result);
assertTrue(result.getMessage().contains("during merge"));
}
}
}
}
@@ -0,0 +1,847 @@
package stirling.software.common.util;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import org.apache.pdfbox.cos.COSDictionary;
import org.apache.pdfbox.cos.COSName;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDPage;
import org.apache.pdfbox.pdmodel.PDResources;
import org.apache.pdfbox.pdmodel.common.PDRectangle;
import org.apache.pdfbox.pdmodel.font.PDType1Font;
import org.apache.pdfbox.pdmodel.font.Standard14Fonts;
import org.apache.pdfbox.pdmodel.interactive.annotation.PDAnnotationWidget;
import org.apache.pdfbox.pdmodel.interactive.form.PDAcroForm;
import org.apache.pdfbox.pdmodel.interactive.form.PDCheckBox;
import org.apache.pdfbox.pdmodel.interactive.form.PDComboBox;
import org.apache.pdfbox.pdmodel.interactive.form.PDListBox;
import org.apache.pdfbox.pdmodel.interactive.form.PDRadioButton;
import org.apache.pdfbox.pdmodel.interactive.form.PDSignatureField;
import org.apache.pdfbox.pdmodel.interactive.form.PDTerminalField;
import org.apache.pdfbox.pdmodel.interactive.form.PDTextField;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test;
/**
* Gap coverage for {@link FormUtils} methods not exercised by {@code FormUtilsTest} (disabled) or
* {@code FormUtilsAdditionalTest}. Focuses on coordinate extraction, the page-map / repair / prune
* / delete / modify lifecycle, and the package-private parsing helpers.
*/
class FormUtilsGapTest {
private record SetupDocument(PDPage page, PDAcroForm acroForm) {}
private static SetupDocument createBasicDocument(PDDocument document) {
PDPage page = new PDPage();
document.addPage(page);
PDAcroForm acroForm = new PDAcroForm(document);
// Register a Helvetica font in the default resources and set a default appearance so
// PDFBox can write text-field values without throwing "/DA is a required entry".
PDResources dr = new PDResources();
dr.put(COSName.getPDFName("Helv"), new PDType1Font(Standard14Fonts.FontName.HELVETICA));
acroForm.setDefaultResources(dr);
acroForm.setDefaultAppearance("/Helv 12 Tf 0 g");
acroForm.setNeedAppearances(true);
document.getDocumentCatalog().setAcroForm(acroForm);
return new SetupDocument(page, acroForm);
}
private static void attachWidget(
SetupDocument setup, PDTerminalField field, PDRectangle rectangle) throws IOException {
PDAnnotationWidget widget = new PDAnnotationWidget();
widget.setRectangle(rectangle);
widget.setPage(setup.page());
// Start from an empty list: a fresh terminal field has no /Kids, so getWidgets() would
// return a synthetic widget wrapping the field dict itself. Re-adding that turns the field
// into a self-referential non-terminal field whose getWidgets() is empty.
List<PDAnnotationWidget> widgets = new ArrayList<>();
widgets.add(widget);
field.setWidgets(widgets);
setup.acroForm().getFields().add(field);
setup.page().getAnnotations().add(widget);
}
// ----------------------------------------------------------------------
// Constants
// ----------------------------------------------------------------------
@Nested
@DisplayName("Field type constants")
class Constants {
@Test
void typeConstantsHaveExpectedValues() {
assertEquals("text", FormUtils.FIELD_TYPE_TEXT);
assertEquals("checkbox", FormUtils.FIELD_TYPE_CHECKBOX);
assertEquals("combobox", FormUtils.FIELD_TYPE_COMBOBOX);
assertEquals("listbox", FormUtils.FIELD_TYPE_LISTBOX);
assertEquals("radio", FormUtils.FIELD_TYPE_RADIO);
assertEquals("button", FormUtils.FIELD_TYPE_BUTTON);
assertEquals("signature", FormUtils.FIELD_TYPE_SIGNATURE);
}
@Test
void choiceFieldTypesContainsExpectedMembers() {
assertTrue(FormUtils.CHOICE_FIELD_TYPES.contains("combobox"));
assertTrue(FormUtils.CHOICE_FIELD_TYPES.contains("listbox"));
assertTrue(FormUtils.CHOICE_FIELD_TYPES.contains("radio"));
assertFalse(FormUtils.CHOICE_FIELD_TYPES.contains("text"));
assertEquals(3, FormUtils.CHOICE_FIELD_TYPES.size());
}
}
// ----------------------------------------------------------------------
// detectFieldType (choice/radio/signature/button branches)
// ----------------------------------------------------------------------
@Nested
@DisplayName("detectFieldType")
class DetectFieldType {
@Test
void comboBoxDetected() throws IOException {
try (PDDocument doc = new PDDocument()) {
SetupDocument setup = createBasicDocument(doc);
assertEquals(
"combobox", FormUtils.detectFieldType(new PDComboBox(setup.acroForm())));
}
}
@Test
void listBoxDetected() throws IOException {
try (PDDocument doc = new PDDocument()) {
SetupDocument setup = createBasicDocument(doc);
assertEquals("listbox", FormUtils.detectFieldType(new PDListBox(setup.acroForm())));
}
}
@Test
void radioButtonDetected() throws IOException {
try (PDDocument doc = new PDDocument()) {
SetupDocument setup = createBasicDocument(doc);
assertEquals(
"radio", FormUtils.detectFieldType(new PDRadioButton(setup.acroForm())));
}
}
@Test
void signatureDetected() throws IOException {
try (PDDocument doc = new PDDocument()) {
SetupDocument setup = createBasicDocument(doc);
assertEquals(
"signature",
FormUtils.detectFieldType(new PDSignatureField(setup.acroForm())));
}
}
}
// ----------------------------------------------------------------------
// isChecked
// ----------------------------------------------------------------------
@Nested
@DisplayName("isChecked")
class IsChecked {
@Test
void nullIsFalse() {
assertFalse(FormUtils.isChecked(null));
}
@Test
void truthyValuesAreChecked() {
assertTrue(FormUtils.isChecked("true"));
assertTrue(FormUtils.isChecked("1"));
assertTrue(FormUtils.isChecked("yes"));
assertTrue(FormUtils.isChecked("on"));
assertTrue(FormUtils.isChecked("checked"));
}
@Test
void truthyValuesAreCaseInsensitiveAndTrimmed() {
assertTrue(FormUtils.isChecked(" TRUE "));
assertTrue(FormUtils.isChecked("Yes"));
assertTrue(FormUtils.isChecked("ON"));
}
@Test
void falsyValuesAreNotChecked() {
assertFalse(FormUtils.isChecked("false"));
assertFalse(FormUtils.isChecked("0"));
assertFalse(FormUtils.isChecked("off"));
assertFalse(FormUtils.isChecked(""));
assertFalse(FormUtils.isChecked("anything"));
}
}
// ----------------------------------------------------------------------
// safeValue
// ----------------------------------------------------------------------
@Test
void safeValueEmptyStringPassesThrough() {
assertEquals("", FormUtils.safeValue(""));
}
// ----------------------------------------------------------------------
// parseMultiChoiceSelections
// ----------------------------------------------------------------------
@Nested
@DisplayName("parseMultiChoiceSelections")
class ParseMultiChoiceSelections {
@Test
void nullReturnsEmpty() {
assertTrue(FormUtils.parseMultiChoiceSelections(null).isEmpty());
}
@Test
void blankReturnsEmpty() {
assertTrue(FormUtils.parseMultiChoiceSelections(" ").isEmpty());
}
@Test
void splitsAndTrims() {
List<String> result = FormUtils.parseMultiChoiceSelections(" a , b ,c ");
assertEquals(List.of("a", "b", "c"), result);
}
@Test
void dropsEmptySegments() {
List<String> result = FormUtils.parseMultiChoiceSelections("a,,b,");
assertEquals(List.of("a", "b"), result);
}
}
// ----------------------------------------------------------------------
// filterChoiceSelections
// ----------------------------------------------------------------------
@Nested
@DisplayName("filterChoiceSelections")
class FilterChoiceSelections {
@Test
void nullSelectionsReturnsEmpty() {
assertTrue(FormUtils.filterChoiceSelections(null, List.of("A"), "f").isEmpty());
}
@Test
void emptySelectionsReturnsEmpty() {
assertTrue(FormUtils.filterChoiceSelections(List.of(), List.of("A"), "f").isEmpty());
}
@Test
void selectionsOfOnlyBlanksReturnsEmpty() {
List<String> selections = new ArrayList<>();
selections.add(" ");
selections.add(null);
assertTrue(FormUtils.filterChoiceSelections(selections, List.of("A"), "f").isEmpty());
}
@Test
void matchingSelectionsAreKeptCaseInsensitively() {
List<String> result =
FormUtils.filterChoiceSelections(
List.of("apple", "BANANA"), List.of("Apple", "Banana", "Cherry"), "f");
// The resolved (canonical) allowed option is returned, not the input.
assertEquals(List.of("Apple", "Banana"), result);
}
@Test
void unsupportedSelectionsAreDropped() {
List<String> result =
FormUtils.filterChoiceSelections(
List.of("Apple", "Grape"), List.of("Apple", "Banana"), "f");
assertEquals(List.of("Apple"), result);
}
@Test
void missingAllowedOptionsThrows() {
org.junit.jupiter.api.Assertions.assertThrows(
IllegalArgumentException.class,
() -> FormUtils.filterChoiceSelections(List.of("Apple"), List.of(), "fieldX"));
}
@Test
void nullAllowedOptionsThrows() {
org.junit.jupiter.api.Assertions.assertThrows(
IllegalArgumentException.class,
() -> FormUtils.filterChoiceSelections(List.of("Apple"), null, "fieldX"));
}
}
// ----------------------------------------------------------------------
// resolveOptions / resolveDisplayOptions / collectChoiceAllowedValues
// ----------------------------------------------------------------------
@Nested
@DisplayName("option resolution")
class OptionResolution {
@Test
void resolveOptionsForComboBox() throws IOException {
try (PDDocument doc = new PDDocument()) {
SetupDocument setup = createBasicDocument(doc);
PDComboBox combo = new PDComboBox(setup.acroForm());
combo.setOptions(List.of("Red", "Green", "Blue"));
List<String> options = FormUtils.resolveOptions(combo);
assertTrue(options.contains("Red"));
assertTrue(options.contains("Green"));
assertTrue(options.contains("Blue"));
}
}
@Test
void resolveOptionsForTextFieldIsEmpty() throws IOException {
try (PDDocument doc = new PDDocument()) {
SetupDocument setup = createBasicDocument(doc);
PDTextField text = new PDTextField(setup.acroForm());
assertTrue(FormUtils.resolveOptions(text).isEmpty());
}
}
@Test
void resolveOptionsForCheckBoxUsesExportValues() throws IOException {
try (PDDocument doc = new PDDocument()) {
SetupDocument setup = createBasicDocument(doc);
PDCheckBox checkBox = new PDCheckBox(setup.acroForm());
checkBox.setExportValues(List.of("Yes"));
assertEquals(List.of("Yes"), FormUtils.resolveOptions(checkBox));
}
}
@Test
void resolveDisplayOptionsEmptyForTextField() throws IOException {
try (PDDocument doc = new PDDocument()) {
SetupDocument setup = createBasicDocument(doc);
PDTextField text = new PDTextField(setup.acroForm());
assertTrue(FormUtils.resolveDisplayOptions(text).isEmpty());
}
}
@Test
void collectChoiceAllowedValuesNullReturnsEmpty() {
assertTrue(FormUtils.collectChoiceAllowedValues(null).isEmpty());
}
@Test
void collectChoiceAllowedValuesReturnsOptions() throws IOException {
try (PDDocument doc = new PDDocument()) {
SetupDocument setup = createBasicDocument(doc);
PDComboBox combo = new PDComboBox(setup.acroForm());
combo.setOptions(List.of("One", "Two"));
List<String> allowed = FormUtils.collectChoiceAllowedValues(combo);
assertTrue(allowed.contains("One"));
assertTrue(allowed.contains("Two"));
}
}
}
// ----------------------------------------------------------------------
// setTextValue
// ----------------------------------------------------------------------
@Nested
@DisplayName("setTextValue")
class SetTextValue {
@Test
void writesValue() throws IOException {
try (PDDocument doc = new PDDocument()) {
SetupDocument setup = createBasicDocument(doc);
PDTextField text = new PDTextField(setup.acroForm());
text.setPartialName("note");
text.setDefaultAppearance("/Helv 12 Tf 0 g");
attachWidget(setup, text, new PDRectangle(20, 600, 200, 20));
FormUtils.setTextValue(text, "hello world");
assertEquals("hello world", text.getValueAsString());
}
}
@Test
void nullValueWritesEmptyString() throws IOException {
try (PDDocument doc = new PDDocument()) {
SetupDocument setup = createBasicDocument(doc);
PDTextField text = new PDTextField(setup.acroForm());
text.setPartialName("note");
text.setDefaultAppearance("/Helv 12 Tf 0 g");
attachWidget(setup, text, new PDRectangle(20, 600, 200, 20));
FormUtils.setTextValue(text, null);
assertEquals("", text.getValueAsString());
}
}
}
// ----------------------------------------------------------------------
// buildFillTemplateRecord (choice branches not covered elsewhere)
// ----------------------------------------------------------------------
@Nested
@DisplayName("buildFillTemplateRecord")
class BuildFillTemplateRecord {
@Test
void comboBoxUsesCurrentValue() {
FormUtils.FormFieldInfo info =
new FormUtils.FormFieldInfo(
"color", "Color", "combobox", "Red", null, false, 0, false, null, 0);
Map<String, Object> result = FormUtils.buildFillTemplateRecord(List.of(info));
assertEquals("Red", result.get("color"));
}
@Test
void singleSelectListBoxUsesValue() {
FormUtils.FormFieldInfo info =
new FormUtils.FormFieldInfo(
"list", "List", "listbox", "Item1", null, false, 0, false, null, 0);
Map<String, Object> result = FormUtils.buildFillTemplateRecord(List.of(info));
assertEquals("Item1", result.get("list"));
}
@Test
void multiSelectListBoxUsesEmptyArray() {
FormUtils.FormFieldInfo info =
new FormUtils.FormFieldInfo(
"list", "List", "listbox", "Item1", null, false, 0, true, null, 0);
Map<String, Object> result = FormUtils.buildFillTemplateRecord(List.of(info));
Object value = result.get("list");
assertTrue(value instanceof List<?>);
assertTrue(((List<?>) value).isEmpty());
}
@Test
void nullValueDefaultsToEmptyString() {
FormUtils.FormFieldInfo info =
new FormUtils.FormFieldInfo(
"name", "Name", "text", null, null, false, 0, false, null, 0);
Map<String, Object> result = FormUtils.buildFillTemplateRecord(List.of(info));
assertEquals("", result.get("name"));
}
@Test
void entriesWithBlankNamesAreSkipped() {
FormUtils.FormFieldInfo blank =
new FormUtils.FormFieldInfo(
" ", "Blank", "text", "x", null, false, 0, false, null, 0);
FormUtils.FormFieldInfo good =
new FormUtils.FormFieldInfo(
"kept", "Kept", "text", "x", null, false, 0, false, null, 0);
Map<String, Object> result = FormUtils.buildFillTemplateRecord(List.of(blank, good));
assertEquals(1, result.size());
assertTrue(result.containsKey("kept"));
}
}
// ----------------------------------------------------------------------
// buildAnnotationPageMap
// ----------------------------------------------------------------------
@Nested
@DisplayName("buildAnnotationPageMap")
class BuildAnnotationPageMap {
@Test
void nullDocumentReturnsEmpty() {
assertTrue(FormUtils.buildAnnotationPageMap(null).isEmpty());
}
@Test
void emptyDocumentReturnsEmpty() throws IOException {
try (PDDocument doc = new PDDocument()) {
doc.addPage(new PDPage());
assertTrue(FormUtils.buildAnnotationPageMap(doc).isEmpty());
}
}
@Test
void mapsWidgetToItsPageIndex() throws IOException {
try (PDDocument doc = new PDDocument()) {
SetupDocument setup = createBasicDocument(doc);
PDTextField text = new PDTextField(setup.acroForm());
text.setPartialName("a");
attachWidget(setup, text, new PDRectangle(10, 10, 100, 20));
Map<COSDictionary, Integer> map = FormUtils.buildAnnotationPageMap(doc);
assertEquals(1, map.size());
assertTrue(map.containsValue(0));
}
}
}
// ----------------------------------------------------------------------
// extractFormFieldsWithCoordinates
// ----------------------------------------------------------------------
@Nested
@DisplayName("extractFormFieldsWithCoordinates")
class ExtractFormFieldsWithCoordinates {
@Test
void nullDocumentReturnsEmpty() {
assertTrue(FormUtils.extractFormFieldsWithCoordinates(null).isEmpty());
}
@Test
void noAcroFormReturnsEmpty() throws IOException {
try (PDDocument doc = new PDDocument()) {
doc.addPage(new PDPage());
assertTrue(FormUtils.extractFormFieldsWithCoordinates(doc).isEmpty());
}
}
@Test
void textFieldProducesWidgetCoordinates() throws IOException {
try (PDDocument doc = new PDDocument()) {
SetupDocument setup = createBasicDocument(doc);
PDTextField text = new PDTextField(setup.acroForm());
text.setPartialName("firstName");
attachWidget(setup, text, new PDRectangle(50, 700, 200, 20));
List<stirling.software.common.model.FormFieldWithCoordinates> fields =
FormUtils.extractFormFieldsWithCoordinates(doc);
assertEquals(1, fields.size());
stirling.software.common.model.FormFieldWithCoordinates field = fields.get(0);
assertEquals("firstName", field.getName());
assertEquals("text", field.getType());
assertNotNull(field.getWidgets());
assertEquals(1, field.getWidgets().size());
stirling.software.common.model.FormFieldWithCoordinates.WidgetCoordinates wc =
field.getWidgets().get(0);
assertEquals(0, wc.getPageIndex());
// x is relative to crop-box origin (0 here), so it equals the lower-left x.
assertEquals(50f, wc.getX(), 0.01f);
assertEquals(200f, wc.getWidth(), 0.01f);
assertEquals(20f, wc.getHeight(), 0.01f);
}
}
@Test
void multipleFieldsAreSortedTopToBottom() throws IOException {
try (PDDocument doc = new PDDocument()) {
SetupDocument setup = createBasicDocument(doc);
PDTextField lower = new PDTextField(setup.acroForm());
lower.setPartialName("lower");
attachWidget(setup, lower, new PDRectangle(50, 100, 200, 20));
PDTextField upper = new PDTextField(setup.acroForm());
upper.setPartialName("upper");
attachWidget(setup, upper, new PDRectangle(50, 700, 200, 20));
List<stirling.software.common.model.FormFieldWithCoordinates> fields =
FormUtils.extractFormFieldsWithCoordinates(doc);
assertEquals(2, fields.size());
// The widget higher on the page (smaller CSS-y after flip) sorts first.
assertEquals("upper", fields.get(0).getName());
assertEquals("lower", fields.get(1).getName());
}
}
}
// ----------------------------------------------------------------------
// repairMissingWidgetPageReferences
// ----------------------------------------------------------------------
@Nested
@DisplayName("repairMissingWidgetPageReferences")
class RepairMissingWidgetPageReferences {
@Test
void nullDocumentDoesNotThrow() {
FormUtils.repairMissingWidgetPageReferences(null);
}
@Test
void documentWithoutAcroFormDoesNotThrow() throws IOException {
try (PDDocument doc = new PDDocument()) {
doc.addPage(new PDPage());
FormUtils.repairMissingWidgetPageReferences(doc);
}
}
@Test
void setsPageReferenceForOrphanWidget() throws IOException {
try (PDDocument doc = new PDDocument()) {
SetupDocument setup = createBasicDocument(doc);
PDTextField text = new PDTextField(setup.acroForm());
text.setPartialName("orphan");
// Build a widget that is on the page's annotation list but has no /P page ref.
PDAnnotationWidget widget = new PDAnnotationWidget();
widget.setRectangle(new PDRectangle(10, 10, 100, 20));
List<PDAnnotationWidget> widgets = new ArrayList<>(text.getWidgets());
widgets.add(widget);
text.setWidgets(widgets);
setup.acroForm().getFields().add(text);
setup.page().getAnnotations().add(widget);
assertNull(widget.getPage());
FormUtils.repairMissingWidgetPageReferences(doc);
assertNotNull(widget.getPage());
}
}
}
// ----------------------------------------------------------------------
// deleteFormFields
// ----------------------------------------------------------------------
@Nested
@DisplayName("deleteFormFields")
class DeleteFormFields {
@Test
void nullDocumentIsNoOp() {
FormUtils.deleteFormFields(null, List.of("a"));
}
@Test
void nullNamesIsNoOp() throws IOException {
try (PDDocument doc = new PDDocument()) {
createBasicDocument(doc);
FormUtils.deleteFormFields(doc, null);
}
}
@Test
void emptyNamesIsNoOp() throws IOException {
try (PDDocument doc = new PDDocument()) {
createBasicDocument(doc);
FormUtils.deleteFormFields(doc, List.of());
}
}
@Test
void removesNamedField() throws IOException {
try (PDDocument doc = new PDDocument()) {
SetupDocument setup = createBasicDocument(doc);
PDTextField keep = new PDTextField(setup.acroForm());
keep.setPartialName("keep");
attachWidget(setup, keep, new PDRectangle(50, 700, 200, 20));
PDTextField remove = new PDTextField(setup.acroForm());
remove.setPartialName("remove");
attachWidget(setup, remove, new PDRectangle(50, 660, 200, 20));
FormUtils.deleteFormFields(doc, List.of("remove"));
List<FormUtils.FormFieldInfo> remaining = FormUtils.extractFormFields(doc);
assertEquals(1, remaining.size());
assertEquals("keep", remaining.get(0).name());
}
}
@Test
void unknownFieldNameIsIgnored() throws IOException {
try (PDDocument doc = new PDDocument()) {
SetupDocument setup = createBasicDocument(doc);
PDTextField keep = new PDTextField(setup.acroForm());
keep.setPartialName("keep");
attachWidget(setup, keep, new PDRectangle(50, 700, 200, 20));
FormUtils.deleteFormFields(doc, List.of("doesNotExist", " ", "keep"));
assertTrue(FormUtils.extractFormFields(doc).isEmpty());
}
}
}
// ----------------------------------------------------------------------
// modifyFormFields
// ----------------------------------------------------------------------
@Nested
@DisplayName("modifyFormFields")
class ModifyFormFields {
@Test
void nullDocumentIsNoOp() {
FormUtils.modifyFormFields(null, List.of());
}
@Test
void nullModificationsIsNoOp() throws IOException {
try (PDDocument doc = new PDDocument()) {
createBasicDocument(doc);
FormUtils.modifyFormFields(doc, null);
}
}
@Test
void emptyModificationsIsNoOp() throws IOException {
try (PDDocument doc = new PDDocument()) {
createBasicDocument(doc);
FormUtils.modifyFormFields(doc, List.of());
}
}
@Test
void inPlaceRenameAndLabelUpdate() throws IOException {
try (PDDocument doc = new PDDocument()) {
SetupDocument setup = createBasicDocument(doc);
PDTextField text = new PDTextField(setup.acroForm());
text.setPartialName("oldName");
text.setDefaultAppearance("/Helv 12 Tf 0 g");
attachWidget(setup, text, new PDRectangle(50, 700, 200, 20));
FormUtils.ModifyFormFieldDefinition mod =
new FormUtils.ModifyFormFieldDefinition(
"oldName",
"newName",
"New Label",
null, // keep type (text) -> in-place path
Boolean.TRUE,
null,
null,
null,
null);
FormUtils.modifyFormFields(doc, List.of(mod));
List<FormUtils.FormFieldInfo> fields = FormUtils.extractFormFields(doc);
assertEquals(1, fields.size());
assertEquals("newName", fields.get(0).name());
assertEquals("New Label", fields.get(0).label());
assertTrue(fields.get(0).required());
}
}
@Test
void unknownTargetIsSkipped() throws IOException {
try (PDDocument doc = new PDDocument()) {
SetupDocument setup = createBasicDocument(doc);
PDTextField text = new PDTextField(setup.acroForm());
text.setPartialName("present");
attachWidget(setup, text, new PDRectangle(50, 700, 200, 20));
FormUtils.ModifyFormFieldDefinition mod =
new FormUtils.ModifyFormFieldDefinition(
"missing", null, null, null, null, null, null, null, null);
FormUtils.modifyFormFields(doc, List.of(mod));
// Untouched field remains.
List<FormUtils.FormFieldInfo> fields = FormUtils.extractFormFields(doc);
assertEquals(1, fields.size());
assertEquals("present", fields.get(0).name());
}
}
@Test
void nullEntriesAndBlankTargetsAreSkipped() throws IOException {
try (PDDocument doc = new PDDocument()) {
SetupDocument setup = createBasicDocument(doc);
PDTextField text = new PDTextField(setup.acroForm());
text.setPartialName("present");
attachWidget(setup, text, new PDRectangle(50, 700, 200, 20));
List<FormUtils.ModifyFormFieldDefinition> mods = new ArrayList<>();
mods.add(null);
mods.add(
new FormUtils.ModifyFormFieldDefinition(
" ", null, null, null, null, null, null, null, null));
FormUtils.modifyFormFields(doc, mods);
assertEquals(1, FormUtils.extractFormFields(doc).size());
}
}
}
// ----------------------------------------------------------------------
// pruneOrphanedFormFields
// ----------------------------------------------------------------------
@Nested
@DisplayName("pruneOrphanedFormFields")
class PruneOrphanedFormFields {
@Test
void nullDocumentIsNoOp() {
FormUtils.pruneOrphanedFormFields(null);
}
@Test
void documentWithoutAcroFormIsNoOp() throws IOException {
try (PDDocument doc = new PDDocument()) {
doc.addPage(new PDPage());
FormUtils.pruneOrphanedFormFields(doc);
assertNull(doc.getDocumentCatalog().getAcroForm(null));
}
}
@Test
void keepsFieldsWithLiveWidgets() throws IOException {
try (PDDocument doc = new PDDocument()) {
SetupDocument setup = createBasicDocument(doc);
PDTextField text = new PDTextField(setup.acroForm());
text.setPartialName("live");
attachWidget(setup, text, new PDRectangle(50, 700, 200, 20));
FormUtils.pruneOrphanedFormFields(doc);
PDAcroForm form = doc.getDocumentCatalog().getAcroForm(null);
assertNotNull(form);
assertEquals(1, form.getFields().size());
}
}
@Test
void dropsAcroFormWhenAllWidgetsOrphaned() throws IOException {
try (PDDocument doc = new PDDocument()) {
SetupDocument setup = createBasicDocument(doc);
PDTextField text = new PDTextField(setup.acroForm());
text.setPartialName("orphan");
attachWidget(setup, text, new PDRectangle(50, 700, 200, 20));
// Remove the widget from the page so it is no longer "live".
setup.page().getAnnotations().clear();
FormUtils.pruneOrphanedFormFields(doc);
assertNull(doc.getDocumentCatalog().getAcroForm(null));
}
}
}
// ----------------------------------------------------------------------
// hasAnyRotatedPage (rotated branch)
// ----------------------------------------------------------------------
@Nested
@DisplayName("hasAnyRotatedPage")
class HasAnyRotatedPage {
@Test
void rotatedPageDetected() throws IOException {
try (PDDocument doc = new PDDocument()) {
PDPage page = new PDPage();
page.setRotation(90);
doc.addPage(page);
assertTrue(FormUtils.hasAnyRotatedPage(doc));
}
}
@Test
void unrotatedPageNotDetected() throws IOException {
try (PDDocument doc = new PDDocument()) {
doc.addPage(new PDPage());
assertFalse(FormUtils.hasAnyRotatedPage(doc));
}
}
}
}
@@ -0,0 +1,333 @@
package stirling.software.common.util;
import static org.junit.jupiter.api.Assertions.*;
import java.io.File;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.List;
import java.util.Map;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import org.springframework.core.io.DefaultResourceLoader;
import org.springframework.core.io.Resource;
import org.springframework.mock.web.MockMultipartFile;
import org.springframework.web.multipart.MultipartFile;
/**
* Gap-coverage tests for {@link GeneralUtils}. Targets the public methods NOT already exercised by
* {@code GeneralUtilsAdditionalTest} (size/url/version/uuid) or {@code GeneralUtilsTest} (filename
* helpers, parsePageList basics, saveKeyToSettings): namely {@code generateFilename}, {@code
* convertToFileName}, {@code evaluateNFunc}, the n-function {@code parsePageList} path, {@code
* createDir}/{@code deleteDirectory}, multipart conversion, the {@code updateSettingsTransactional}
* early-return guards, {@code getResourcesFromLocationPattern}, and the environment helpers {@code
* generateMachineFingerprint}/{@code getLocalNetworkIp}.
*/
class GeneralUtilsGapTest {
@Nested
@DisplayName("generateFilename")
class GenerateFilenameTests {
@Test
@DisplayName("removes extension then appends suffix")
void removesAndAppends() {
assertEquals(
"report_out.pdf", GeneralUtils.generateFilename("report.docx", "_out.pdf"));
}
@Test
@DisplayName("null filename uses default base")
void nullFilename() {
assertEquals("default_out.pdf", GeneralUtils.generateFilename(null, "_out.pdf"));
}
@Test
@DisplayName("filename without extension is preserved")
void noExtension() {
assertEquals("README_x", GeneralUtils.generateFilename("README", "_x"));
}
}
@Nested
@DisplayName("convertToFileName")
class ConvertToFileNameTests {
@Test
@DisplayName("null returns underscore")
void nullReturnsUnderscore() {
assertEquals("_", GeneralUtils.convertToFileName(null));
}
@Test
@DisplayName("keeps letters and digits, replaces others with underscore")
void replacesUnsafeChars() {
assertEquals("my_file_2024_", GeneralUtils.convertToFileName("my file/2024!"));
}
@Test
@DisplayName("alphanumeric input is unchanged")
void alphanumericUnchanged() {
assertEquals("File123", GeneralUtils.convertToFileName("File123"));
}
@Test
@DisplayName("truncates to 50 characters")
void truncatesToFifty() {
String input = "a".repeat(100);
assertEquals(50, GeneralUtils.convertToFileName(input).length());
}
}
@Nested
@DisplayName("evaluateNFunc")
class EvaluateNFuncTests {
@Test
@DisplayName("null expression throws")
void nullThrows() {
assertThrows(
IllegalArgumentException.class, () -> GeneralUtils.evaluateNFunc(null, 10));
}
@Test
@DisplayName("blank expression throws")
void blankThrows() {
assertThrows(
IllegalArgumentException.class, () -> GeneralUtils.evaluateNFunc(" ", 10));
}
@Test
@DisplayName("maxValue below 1 throws")
void maxValueTooLow() {
assertThrows(IllegalArgumentException.class, () -> GeneralUtils.evaluateNFunc("n", 0));
}
@Test
@DisplayName("maxValue above 10000 throws")
void maxValueTooHigh() {
assertThrows(
IllegalArgumentException.class, () -> GeneralUtils.evaluateNFunc("n", 10001));
}
@Test
@DisplayName("invalid characters throw")
void invalidCharsThrow() {
assertThrows(
IllegalArgumentException.class, () -> GeneralUtils.evaluateNFunc("n$", 10));
}
@Test
@DisplayName("identity 'n' yields all pages up to maxValue")
void identity() {
assertEquals(List.of(1, 2, 3, 4, 5), GeneralUtils.evaluateNFunc("n", 5));
}
@Test
@DisplayName("2n yields even values within bounds")
void doubling() {
assertEquals(List.of(2, 4, 6), GeneralUtils.evaluateNFunc("2n", 6));
}
@Test
@DisplayName("implicit multiplication 'n(n-1)' is handled")
void implicitMultiplication() {
// n*(n-1): n=1->0(excluded), n=2->2, n=3->6 ; capped at maxValue 6
assertEquals(List.of(2, 6), GeneralUtils.evaluateNFunc("n(n-1)", 6));
}
@Test
@DisplayName("results outside (0, maxValue] are excluded")
void boundsExcluded() {
// n+10 always exceeds maxValue 5 -> empty
assertTrue(GeneralUtils.evaluateNFunc("n+10", 5).isEmpty());
}
}
@Nested
@DisplayName("parsePageList n-function path")
class ParsePageListNFuncTests {
@Test
@DisplayName("n-function token expands to matching one-based pages")
void nFunctionOneBased() {
// 2n for total 6 (one-based) -> values 2,4,6 mapped to (v-1+1)=v
assertEquals(List.of(2, 4, 6), GeneralUtils.parsePageList("2n", 6, true));
}
@Test
@DisplayName("n-function token zero-based subtracts one")
void nFunctionZeroBased() {
// 2n for total 6 zero-based -> values 2,4,6 mapped to (v-1+0)=v-1
assertEquals(List.of(1, 3, 5), GeneralUtils.parsePageList("2n", 6, false));
}
}
@Nested
@DisplayName("createDir and deleteDirectory")
class DirectoryTests {
@Test
@DisplayName("createDir makes a nested directory and returns true")
void createNested(@TempDir Path tempDir) {
Path nested = tempDir.resolve("a").resolve("b").resolve("c");
assertTrue(GeneralUtils.createDir(nested.toString()));
assertTrue(Files.isDirectory(nested));
}
@Test
@DisplayName("createDir returns true when directory already exists")
void createExisting(@TempDir Path tempDir) {
assertTrue(GeneralUtils.createDir(tempDir.toString()));
}
@Test
@DisplayName("deleteDirectory removes a populated tree without touching siblings")
void deletePopulatedTree(@TempDir Path tempDir) throws IOException {
Path sibling = tempDir.resolve("sibling");
Files.createDirectories(sibling);
Files.writeString(sibling.resolve("keep.txt"), "data");
Path root = tempDir.resolve("root");
Files.createDirectories(root.resolve("nested"));
Files.writeString(root.resolve("a.txt"), "x");
Files.writeString(root.resolve("nested").resolve("b.txt"), "y");
GeneralUtils.deleteDirectory(root);
assertFalse(Files.exists(root));
assertTrue(Files.exists(sibling.resolve("keep.txt")));
}
}
@Nested
@DisplayName("multipart conversion")
class MultipartTests {
@Test
@DisplayName("convertMultipartFileToFile writes content to a temp file")
void convertWritesContent() throws IOException {
byte[] content = "hello world".getBytes(StandardCharsets.UTF_8);
MultipartFile mf =
new MockMultipartFile("file", "input.bin", "application/octet-stream", content);
File out = GeneralUtils.convertMultipartFileToFile(mf);
try {
assertTrue(out.exists());
assertArrayEquals(content, Files.readAllBytes(out.toPath()));
} finally {
Files.deleteIfExists(out.toPath());
}
}
@Test
@DisplayName("convertMultipartFileToFile handles empty input")
void convertEmpty() throws IOException {
MultipartFile mf =
new MockMultipartFile(
"file", "empty.bin", "application/octet-stream", new byte[0]);
File out = GeneralUtils.convertMultipartFileToFile(mf);
try {
assertTrue(out.exists());
assertEquals(0, out.length());
} finally {
Files.deleteIfExists(out.toPath());
}
}
@Test
@DisplayName("multipartToFile writes content to a .pdf temp file")
void multipartToFileWritesContent() throws IOException {
byte[] content = "%PDF-1.7 minimal".getBytes(StandardCharsets.UTF_8);
MultipartFile mf = new MockMultipartFile("file", "doc.pdf", "application/pdf", content);
File out = GeneralUtils.multipartToFile(mf);
try {
assertTrue(out.exists());
assertTrue(out.getName().endsWith(".pdf"));
assertArrayEquals(content, Files.readAllBytes(out.toPath()));
} finally {
Files.deleteIfExists(out.toPath());
}
}
}
@Nested
@DisplayName("getResourcesFromLocationPattern")
class ResourcePatternTests {
@Test
@DisplayName("file: pattern resolves matching files in a directory")
void filePatternResolves(@TempDir Path tempDir) throws Exception {
Files.writeString(tempDir.resolve("one.txt"), "1");
Files.writeString(tempDir.resolve("two.txt"), "2");
String pattern = "file:" + tempDir.toString().replace("\\", "/") + "/*";
Resource[] resources =
GeneralUtils.getResourcesFromLocationPattern(
pattern, new DefaultResourceLoader());
assertNotNull(resources);
assertEquals(2, resources.length);
}
@Test
@DisplayName("classpath pattern with no matches returns an empty array")
void classpathNoMatches() throws Exception {
Resource[] resources =
GeneralUtils.getResourcesFromLocationPattern(
"classpath*:this/path/does/not/exist/**/*.nope",
new DefaultResourceLoader());
assertNotNull(resources);
assertEquals(0, resources.length);
}
}
@Nested
@DisplayName("updateSettingsTransactional early-return guards")
class SettingsGuardTests {
@Test
@DisplayName("null map returns without throwing")
void nullMapNoOp() {
assertDoesNotThrow(() -> GeneralUtils.updateSettingsTransactional(null));
}
@Test
@DisplayName("empty map returns without throwing")
void emptyMapNoOp() {
assertDoesNotThrow(() -> GeneralUtils.updateSettingsTransactional(Map.of()));
}
}
@Nested
@DisplayName("environment-dependent helpers")
class EnvironmentHelperTests {
@Test
@DisplayName("generateMachineFingerprint returns a non-blank, deterministic value")
void fingerprintStable() {
String first = GeneralUtils.generateMachineFingerprint();
assertNotNull(first);
assertFalse(first.isBlank());
// Deterministic within the same JVM/host
assertEquals(first, GeneralUtils.generateMachineFingerprint());
}
@Test
@DisplayName("getLocalNetworkIp returns null or a dotted IPv4 string")
void localIpFormat() {
String ip = GeneralUtils.getLocalNetworkIp();
if (ip != null) {
assertTrue(ip.matches("\\d{1,3}(\\.\\d{1,3}){3}"), "unexpected IP form: " + ip);
}
}
}
}
@@ -0,0 +1,370 @@
package stirling.software.common.util;
import static org.junit.jupiter.api.Assertions.assertArrayEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.mockito.ArgumentMatchers.eq;
import static org.mockito.Mockito.lenient;
import static org.mockito.Mockito.mock;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.util.HashMap;
import java.util.LinkedHashMap;
import java.util.Map;
import java.util.zip.ZipEntry;
import java.util.zip.ZipInputStream;
import java.util.zip.ZipOutputStream;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.service.SsrfProtectionService;
class OfficeDocumentSanitizerTest {
private static final String EXTERNAL_URL = "https://webhook.site/ssrf-callback";
private static final String INTERNAL_TARGET = "media/image1.png";
private static final String DOCX_RELS =
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>"
+ "<Relationships xmlns=\"http://schemas.openxmlformats.org/package/2006/relationships\">"
+ "<Relationship Id=\"rId1\" Type=\"http://schemas.openxmlformats.org/officeDocument/2006/relationships/image\""
+ " Target=\""
+ EXTERNAL_URL
+ "\" TargetMode=\"External\"/>"
+ "<Relationship Id=\"rId2\" Type=\"http://schemas.openxmlformats.org/officeDocument/2006/relationships/image\""
+ " Target=\""
+ INTERNAL_TARGET
+ "\"/>"
+ "</Relationships>";
private static final String DOCX_DOCUMENT =
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>"
+ "<w:document xmlns:w=\"http://schemas.openxmlformats.org/wordprocessingml/2006/main\">"
+ "<w:body><w:p/></w:body></w:document>";
private static final String ODF_CONTENT_EXTERNAL =
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>"
+ "<office:document-content"
+ " xmlns:office=\"urn:oasis:names:tc:opendocument:xmlns:office:1.0\""
+ " xmlns:draw=\"urn:oasis:names:tc:opendocument:xmlns:drawing:1.0\""
+ " xmlns:xlink=\"http://www.w3.org/1999/xlink\">"
+ "<office:body><office:text>"
+ "<draw:frame><draw:image xlink:href=\""
+ EXTERNAL_URL
+ "\" xlink:type=\"simple\"/></draw:frame>"
+ "<draw:frame><draw:image xlink:href=\"Pictures/image1.png\" xlink:type=\"simple\"/></draw:frame>"
+ "</office:text></office:body></office:document-content>";
private SsrfProtectionService ssrfProtectionService;
private ApplicationProperties applicationProperties;
private OfficeDocumentSanitizer sanitizer;
@BeforeEach
void setUp() {
applicationProperties = new ApplicationProperties();
ssrfProtectionService = mock(SsrfProtectionService.class);
sanitizer = new OfficeDocumentSanitizer(ssrfProtectionService, applicationProperties);
}
@Test
void isSanitizableExtension_recognizesOoxmlAndOdf() {
assertTrue(sanitizer.isSanitizableExtension("docx"));
assertTrue(sanitizer.isSanitizableExtension("DOCX"));
assertTrue(sanitizer.isSanitizableExtension("xlsx"));
assertTrue(sanitizer.isSanitizableExtension("pptx"));
assertTrue(sanitizer.isSanitizableExtension("odt"));
assertTrue(sanitizer.isSanitizableExtension("ods"));
assertTrue(sanitizer.isSanitizableExtension("odp"));
assertFalse(sanitizer.isSanitizableExtension("pdf"));
assertFalse(sanitizer.isSanitizableExtension("html"));
assertFalse(sanitizer.isSanitizableExtension(""));
assertFalse(sanitizer.isSanitizableExtension(null));
}
@Test
void sanitize_stripsOoxmlExternalRelationship() throws IOException {
Map<String, byte[]> entries = new LinkedHashMap<>();
entries.put("word/_rels/document.xml.rels", DOCX_RELS.getBytes(StandardCharsets.UTF_8));
entries.put("word/document.xml", DOCX_DOCUMENT.getBytes(StandardCharsets.UTF_8));
byte[] docx = zip(entries);
byte[] cleaned = sanitizer.sanitize(docx, "docx");
Map<String, byte[]> result = unzip(cleaned);
String rels =
new String(result.get("word/_rels/document.xml.rels"), StandardCharsets.UTF_8);
assertFalse(rels.contains(EXTERNAL_URL), "External URL should be stripped from .rels");
assertFalse(
rels.toLowerCase().contains("targetmode=\"external\""),
"TargetMode=External relationship should be removed");
assertTrue(rels.contains(INTERNAL_TARGET), "Internal image target should be preserved");
assertArrayEquals(
DOCX_DOCUMENT.getBytes(StandardCharsets.UTF_8),
result.get("word/document.xml"),
"Non-rels entries must be untouched");
}
@Test
void sanitize_pptxExternalImageRelStripped() throws IOException {
String pptxRels =
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>"
+ "<Relationships xmlns=\"http://schemas.openxmlformats.org/package/2006/relationships\">"
+ "<Relationship Id=\"rId1\" Type=\"http://schemas.openxmlformats.org/officeDocument/2006/relationships/image\""
+ " Target=\""
+ EXTERNAL_URL
+ "\" TargetMode=\"External\"/>"
+ "</Relationships>";
Map<String, byte[]> entries = new LinkedHashMap<>();
entries.put("ppt/slides/_rels/slide1.xml.rels", pptxRels.getBytes(StandardCharsets.UTF_8));
byte[] pptx = zip(entries);
byte[] cleaned = sanitizer.sanitize(pptx, "pptx");
Map<String, byte[]> result = unzip(cleaned);
String rels =
new String(result.get("ppt/slides/_rels/slide1.xml.rels"), StandardCharsets.UTF_8);
assertFalse(rels.contains(EXTERNAL_URL));
}
@Test
void sanitize_xlsxExternalImageRelStripped() throws IOException {
String xlsxRels =
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>"
+ "<Relationships xmlns=\"http://schemas.openxmlformats.org/package/2006/relationships\">"
+ "<Relationship Id=\"rId1\" Type=\"http://schemas.openxmlformats.org/officeDocument/2006/relationships/image\""
+ " Target=\""
+ EXTERNAL_URL
+ "\" TargetMode=\"External\"/>"
+ "</Relationships>";
Map<String, byte[]> entries = new LinkedHashMap<>();
entries.put(
"xl/drawings/_rels/drawing1.xml.rels", xlsxRels.getBytes(StandardCharsets.UTF_8));
byte[] xlsx = zip(entries);
byte[] cleaned = sanitizer.sanitize(xlsx, "xlsx");
Map<String, byte[]> result = unzip(cleaned);
String rels =
new String(
result.get("xl/drawings/_rels/drawing1.xml.rels"), StandardCharsets.UTF_8);
assertFalse(rels.contains(EXTERNAL_URL));
}
@Test
void sanitize_odtStripsExternalXlinkHrefButKeepsInternal() throws IOException {
Map<String, byte[]> entries = new LinkedHashMap<>();
entries.put("content.xml", ODF_CONTENT_EXTERNAL.getBytes(StandardCharsets.UTF_8));
String manifestXml =
"<?xml version=\"1.0\"?><manifest:manifest"
+ " xmlns:manifest=\"urn:oasis:names:tc:opendocument:xmlns:manifest:1.0\"/>";
entries.put("META-INF/manifest.xml", manifestXml.getBytes(StandardCharsets.UTF_8));
byte[] odt = zip(entries);
byte[] cleaned = sanitizer.sanitize(odt, "odt");
Map<String, byte[]> result = unzip(cleaned);
String content = new String(result.get("content.xml"), StandardCharsets.UTF_8);
assertFalse(content.contains(EXTERNAL_URL), "External xlink:href should be stripped");
assertTrue(content.contains("Pictures/image1.png"), "Internal href should be preserved");
}
@Test
void sanitize_odsStripsExternalXlinkHref() throws IOException {
Map<String, byte[]> entries = new LinkedHashMap<>();
entries.put("content.xml", ODF_CONTENT_EXTERNAL.getBytes(StandardCharsets.UTF_8));
byte[] ods = zip(entries);
byte[] cleaned = sanitizer.sanitize(ods, "ods");
Map<String, byte[]> result = unzip(cleaned);
String content = new String(result.get("content.xml"), StandardCharsets.UTF_8);
assertFalse(content.contains(EXTERNAL_URL));
}
@Test
void sanitize_odpStripsExternalXlinkHrefInStylesXml() throws IOException {
String stylesXml =
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>"
+ "<office:document-styles"
+ " xmlns:office=\"urn:oasis:names:tc:opendocument:xmlns:office:1.0\""
+ " xmlns:draw=\"urn:oasis:names:tc:opendocument:xmlns:drawing:1.0\""
+ " xmlns:xlink=\"http://www.w3.org/1999/xlink\">"
+ "<draw:image xlink:href=\""
+ EXTERNAL_URL
+ "\"/></office:document-styles>";
Map<String, byte[]> entries = new LinkedHashMap<>();
entries.put("styles.xml", stylesXml.getBytes(StandardCharsets.UTF_8));
byte[] odp = zip(entries);
byte[] cleaned = sanitizer.sanitize(odp, "odp");
Map<String, byte[]> result = unzip(cleaned);
String content = new String(result.get("styles.xml"), StandardCharsets.UTF_8);
assertFalse(content.contains(EXTERNAL_URL));
}
@Test
void sanitize_disabledByConfigReturnsOriginal() throws IOException {
applicationProperties.getSystem().setDisableSanitize(true);
Map<String, byte[]> entries = new LinkedHashMap<>();
entries.put("word/_rels/document.xml.rels", DOCX_RELS.getBytes(StandardCharsets.UTF_8));
byte[] docx = zip(entries);
byte[] result = sanitizer.sanitize(docx, "docx");
assertArrayEquals(docx, result);
}
@Test
void sanitize_unrecognizedExtensionReturnsOriginal() throws IOException {
byte[] original = "irrelevant".getBytes(StandardCharsets.UTF_8);
byte[] result = sanitizer.sanitize(original, "pdf");
assertArrayEquals(original, result);
}
@Test
void sanitize_emptyInputThrows() {
assertThrows(IOException.class, () -> sanitizer.sanitize(new byte[0], "docx"));
}
@Test
void sanitize_nullInputThrows() {
assertThrows(IOException.class, () -> sanitizer.sanitize(null, "docx"));
}
@Test
void sanitize_preservesEntryWithExternalRefWhenAdminAllowsDomain() throws IOException {
applicationProperties
.getSystem()
.getHtml()
.getUrlSecurity()
.getAllowedDomains()
.add("webhook.site");
lenient().when(ssrfProtectionService.isUrlAllowed(eq(EXTERNAL_URL))).thenReturn(true);
Map<String, byte[]> entries = new LinkedHashMap<>();
entries.put("word/_rels/document.xml.rels", DOCX_RELS.getBytes(StandardCharsets.UTF_8));
byte[] docx = zip(entries);
byte[] cleaned = sanitizer.sanitize(docx, "docx");
Map<String, byte[]> result = unzip(cleaned);
String rels =
new String(result.get("word/_rels/document.xml.rels"), StandardCharsets.UTF_8);
assertTrue(rels.contains(EXTERNAL_URL), "Allow-listed external URL should be preserved");
}
@Test
void sanitize_doesNotConsultSsrfServiceWhenAllowedDomainsEmpty() throws IOException {
// Even if mock would say allowed, we should not invoke it when there is no allow-list,
// because MEDIUM default would let public URLs through and re-introduce the vulnerability.
lenient().when(ssrfProtectionService.isUrlAllowed(eq(EXTERNAL_URL))).thenReturn(true);
Map<String, byte[]> entries = new LinkedHashMap<>();
entries.put("word/_rels/document.xml.rels", DOCX_RELS.getBytes(StandardCharsets.UTF_8));
byte[] docx = zip(entries);
byte[] cleaned = sanitizer.sanitize(docx, "docx");
Map<String, byte[]> result = unzip(cleaned);
String rels =
new String(result.get("word/_rels/document.xml.rels"), StandardCharsets.UTF_8);
assertFalse(rels.contains(EXTERNAL_URL));
}
@Test
void sanitize_handlesNonXmlEntriesSafely() throws IOException {
Map<String, byte[]> entries = new LinkedHashMap<>();
byte[] imageBytes = new byte[] {(byte) 0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a};
entries.put("word/media/image1.png", imageBytes);
entries.put("word/_rels/document.xml.rels", DOCX_RELS.getBytes(StandardCharsets.UTF_8));
byte[] docx = zip(entries);
byte[] cleaned = sanitizer.sanitize(docx, "docx");
Map<String, byte[]> result = unzip(cleaned);
assertArrayEquals(imageBytes, result.get("word/media/image1.png"));
}
@Test
void sanitize_internalLinksKeptWhenNoExternalPresent() throws IOException {
String internalOnlyRels =
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>"
+ "<Relationships xmlns=\"http://schemas.openxmlformats.org/package/2006/relationships\">"
+ "<Relationship Id=\"rId1\" Type=\"http://schemas.openxmlformats.org/officeDocument/2006/relationships/image\""
+ " Target=\"media/image1.png\"/>"
+ "</Relationships>";
Map<String, byte[]> entries = new LinkedHashMap<>();
entries.put(
"word/_rels/document.xml.rels", internalOnlyRels.getBytes(StandardCharsets.UTF_8));
byte[] docx = zip(entries);
byte[] cleaned = sanitizer.sanitize(docx, "docx");
Map<String, byte[]> result = unzip(cleaned);
String rels =
new String(result.get("word/_rels/document.xml.rels"), StandardCharsets.UTF_8);
assertTrue(rels.contains("media/image1.png"));
}
@Test
void sanitize_corruptZipProducesSafeOutput() throws IOException {
byte[] garbage = "this is not a zip file".getBytes(StandardCharsets.UTF_8);
byte[] result = sanitizer.sanitize(garbage, "docx");
Map<String, byte[]> entries = unzip(result);
assertTrue(entries.isEmpty(), "Garbage input must not yield exploitable entries");
}
@Test
void sanitize_relativeOdfPathsArePreserved() throws IOException {
String content =
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>"
+ "<office:document-content"
+ " xmlns:office=\"urn:oasis:names:tc:opendocument:xmlns:office:1.0\""
+ " xmlns:draw=\"urn:oasis:names:tc:opendocument:xmlns:drawing:1.0\""
+ " xmlns:xlink=\"http://www.w3.org/1999/xlink\">"
+ "<draw:image xlink:href=\"../Pictures/image1.png\"/>"
+ "<draw:image xlink:href=\"#anchor\"/>"
+ "</office:document-content>";
Map<String, byte[]> entries = new LinkedHashMap<>();
entries.put("content.xml", content.getBytes(StandardCharsets.UTF_8));
byte[] odt = zip(entries);
byte[] cleaned = sanitizer.sanitize(odt, "odt");
Map<String, byte[]> result = unzip(cleaned);
String out = new String(result.get("content.xml"), StandardCharsets.UTF_8);
assertTrue(out.contains("../Pictures/image1.png"));
assertTrue(out.contains("#anchor"));
}
private static byte[] zip(Map<String, byte[]> entries) throws IOException {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
try (ZipOutputStream zos = new ZipOutputStream(baos)) {
for (Map.Entry<String, byte[]> e : entries.entrySet()) {
ZipEntry entry = new ZipEntry(e.getKey());
zos.putNextEntry(entry);
zos.write(e.getValue());
zos.closeEntry();
}
}
return baos.toByteArray();
}
private static Map<String, byte[]> unzip(byte[] data) throws IOException {
Map<String, byte[]> entries = new HashMap<>();
try (ZipInputStream zis = new ZipInputStream(new ByteArrayInputStream(data))) {
ZipEntry e;
while ((e = zis.getNextEntry()) != null) {
entries.put(e.getName(), zis.readAllBytes());
zis.closeEntry();
}
}
return entries;
}
}
@@ -0,0 +1,446 @@
package stirling.software.common.util;
import static org.junit.jupiter.api.Assertions.*;
import static org.mockito.Mockito.*;
import java.io.ByteArrayOutputStream;
import java.nio.charset.StandardCharsets;
import java.time.ZoneId;
import java.time.ZonedDateTime;
import java.util.ArrayList;
import java.util.Base64;
import java.util.Date;
import java.util.List;
import java.util.Map;
import org.apache.pdfbox.Loader;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDDocumentNameDictionary;
import org.apache.pdfbox.pdmodel.PDEmbeddedFilesNameTreeNode;
import org.apache.pdfbox.pdmodel.PDPage;
import org.apache.pdfbox.pdmodel.PDPageContentStream;
import org.apache.pdfbox.pdmodel.common.PDRectangle;
import org.apache.pdfbox.pdmodel.common.filespecification.PDComplexFileSpecification;
import org.apache.pdfbox.pdmodel.font.PDType1Font;
import org.apache.pdfbox.pdmodel.font.Standard14Fonts;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import org.mockito.junit.jupiter.MockitoSettings;
import org.mockito.quality.Strictness;
import stirling.software.common.service.CustomPDFDocumentFactory;
@ExtendWith(MockitoExtension.class)
@MockitoSettings(strictness = Strictness.LENIENT)
class PdfAttachmentHandlerGapTest {
@Mock private CustomPDFDocumentFactory pdfDocumentFactory;
// ----- helpers -------------------------------------------------------
/** Builds a tiny one-page PDF whose page renders the given text lines, each on its own line. */
private static byte[] pdfWithLines(String... lines) throws Exception {
try (PDDocument doc = new PDDocument()) {
PDPage page = new PDPage(PDRectangle.A4);
doc.addPage(page);
try (PDPageContentStream cs = new PDPageContentStream(doc, page)) {
cs.setFont(new PDType1Font(Standard14Fonts.FontName.HELVETICA), 12);
float y = 720f;
for (String line : lines) {
cs.beginText();
cs.newLineAtOffset(72f, y);
cs.showText(line);
cs.endText();
y -= 20f;
}
}
ByteArrayOutputStream baos = new ByteArrayOutputStream();
doc.save(baos);
return baos.toByteArray();
}
}
/** Builds a blank one-page PDF with no text. */
private static byte[] blankPdf() throws Exception {
try (PDDocument doc = new PDDocument()) {
doc.addPage(new PDPage(PDRectangle.A4));
ByteArrayOutputStream baos = new ByteArrayOutputStream();
doc.save(baos);
return baos.toByteArray();
}
}
private static EmlParser.EmailAttachment attachment(String filename, byte[] data) {
EmlParser.EmailAttachment a = new EmlParser.EmailAttachment();
a.setFilename(filename);
a.setData(data);
a.setContentType("application/pdf");
return a;
}
private static List<String> embeddedFileNames(byte[] pdfBytes) throws Exception {
List<String> names = new ArrayList<>();
try (PDDocument doc = Loader.loadPDF(pdfBytes)) {
PDDocumentNameDictionary docNames = doc.getDocumentCatalog().getNames();
if (docNames == null) {
return names;
}
PDEmbeddedFilesNameTreeNode tree = docNames.getEmbeddedFiles();
if (tree == null) {
return names;
}
Map<String, PDComplexFileSpecification> map = tree.getNames();
if (map != null) {
names.addAll(map.keySet());
}
}
return names;
}
// ----- attachFilesToPdf: short-circuit branches ----------------------
@Nested
@DisplayName("attachFilesToPdf short-circuit handling")
class ShortCircuitTests {
@Test
@DisplayName("null attachment list returns the original bytes untouched")
void nullAttachments_returnsOriginalBytes() throws Exception {
byte[] original = {1, 2, 3, 4};
byte[] result =
PdfAttachmentHandler.attachFilesToPdf(original, null, pdfDocumentFactory);
assertSame(original, result);
verifyNoInteractions(pdfDocumentFactory);
}
@Test
@DisplayName("empty attachment list returns the original bytes untouched")
void emptyAttachments_returnsOriginalBytes() throws Exception {
byte[] original = {9, 8, 7};
byte[] result =
PdfAttachmentHandler.attachFilesToPdf(
original, new ArrayList<>(), pdfDocumentFactory);
assertSame(original, result);
verifyNoInteractions(pdfDocumentFactory);
}
@Test
@DisplayName("attachments with no usable data are skipped and a clean PDF is returned")
void attachmentsWithoutData_produceNoEmbeddedFiles() throws Exception {
byte[] pdfBytes = blankPdf();
when(pdfDocumentFactory.load(pdfBytes)).thenReturn(Loader.loadPDF(pdfBytes));
List<EmlParser.EmailAttachment> attachments = new ArrayList<>();
attachments.add(attachment("empty.pdf", new byte[0]));
attachments.add(attachment("alsoEmpty.pdf", null));
byte[] result =
PdfAttachmentHandler.attachFilesToPdf(
pdfBytes, attachments, pdfDocumentFactory);
assertNotNull(result);
assertTrue(result.length > 0);
assertTrue(embeddedFileNames(result).isEmpty());
}
}
// ----- attachFilesToPdf: embedding happy paths -----------------------
@Nested
@DisplayName("attachFilesToPdf embedding behaviour")
class EmbeddingTests {
@Test
@DisplayName("embeds attachment data even when no '@' marker exists in the PDF text")
void embedsAttachment_withoutMarker() throws Exception {
byte[] pdfBytes = pdfWithLines("Just a plain document with no attachment markers");
when(pdfDocumentFactory.load(pdfBytes)).thenReturn(Loader.loadPDF(pdfBytes));
List<EmlParser.EmailAttachment> attachments = new ArrayList<>();
attachments.add(attachment("report.pdf", "hello".getBytes(StandardCharsets.UTF_8)));
byte[] result =
PdfAttachmentHandler.attachFilesToPdf(
pdfBytes, attachments, pdfDocumentFactory);
List<String> embedded = embeddedFileNames(result);
assertEquals(1, embedded.size());
assertTrue(embedded.contains("report.pdf"));
}
@Test
@DisplayName("embeds attachment and adds an annotation when an '@' marker matches")
void embedsAttachment_withMatchingMarker() throws Exception {
byte[] pdfBytes =
pdfWithLines("Email body text here", "Attachments (1)", "@report.pdf (5 KB)");
when(pdfDocumentFactory.load(pdfBytes)).thenReturn(Loader.loadPDF(pdfBytes));
List<EmlParser.EmailAttachment> attachments = new ArrayList<>();
attachments.add(attachment("report.pdf", "PDFDATA".getBytes(StandardCharsets.UTF_8)));
byte[] result =
PdfAttachmentHandler.attachFilesToPdf(
pdfBytes, attachments, pdfDocumentFactory);
List<String> embedded = embeddedFileNames(result);
assertTrue(embedded.contains("report.pdf"));
// The annotation pass should have run and produced at least one annotation on the
// page that contains the marker (a blank source page has none).
try (PDDocument doc = Loader.loadPDF(result)) {
assertFalse(doc.getPage(0).getAnnotations().isEmpty());
}
}
@Test
@DisplayName("attachment without a filename falls back to a generated embedded name")
void embedsAttachment_withGeneratedName() throws Exception {
byte[] pdfBytes = blankPdf();
when(pdfDocumentFactory.load(pdfBytes)).thenReturn(Loader.loadPDF(pdfBytes));
EmlParser.EmailAttachment a = new EmlParser.EmailAttachment();
a.setFilename(null);
a.setData("x".getBytes(StandardCharsets.UTF_8));
List<EmlParser.EmailAttachment> attachments = new ArrayList<>();
attachments.add(a);
byte[] result =
PdfAttachmentHandler.attachFilesToPdf(
pdfBytes, attachments, pdfDocumentFactory);
// A single embedded file should exist with a non-blank generated name.
List<String> embedded = embeddedFileNames(result);
assertEquals(1, embedded.size());
assertFalse(embedded.get(0).isBlank());
}
@Test
@DisplayName("duplicate attachment filenames produce uniquely named embedded files")
void embedsAttachments_withDuplicateNames() throws Exception {
byte[] pdfBytes = blankPdf();
when(pdfDocumentFactory.load(pdfBytes)).thenReturn(Loader.loadPDF(pdfBytes));
List<EmlParser.EmailAttachment> attachments = new ArrayList<>();
attachments.add(attachment("dup.pdf", "a".getBytes(StandardCharsets.UTF_8)));
attachments.add(attachment("dup.pdf", "b".getBytes(StandardCharsets.UTF_8)));
byte[] result =
PdfAttachmentHandler.attachFilesToPdf(
pdfBytes, attachments, pdfDocumentFactory);
List<String> embedded = embeddedFileNames(result);
assertEquals(2, embedded.size());
assertTrue(embedded.contains("dup.pdf"));
// The second one must have been disambiguated, not overwritten.
assertTrue(embedded.stream().anyMatch(n -> !"dup.pdf".equals(n)));
}
}
// ----- attachFilesToPdf: error wrapping ------------------------------
@Nested
@DisplayName("attachFilesToPdf error handling")
class ErrorHandlingTests {
@Test
@DisplayName("IOException from the factory load propagates to the caller")
void factoryIOException_propagates() throws Exception {
byte[] pdfBytes = {0x25, 0x50, 0x44, 0x46}; // "%PDF"
when(pdfDocumentFactory.load(pdfBytes))
.thenThrow(new java.io.IOException("boom from factory"));
List<EmlParser.EmailAttachment> attachments = new ArrayList<>();
attachments.add(attachment("a.pdf", "data".getBytes(StandardCharsets.UTF_8)));
java.io.IOException ex =
assertThrows(
java.io.IOException.class,
() ->
PdfAttachmentHandler.attachFilesToPdf(
pdfBytes, attachments, pdfDocumentFactory));
assertTrue(ex.getMessage().contains("boom from factory"));
}
}
// ----- AttachmentMarkerPositionFinder --------------------------------
@Nested
@DisplayName("AttachmentMarkerPositionFinder")
class MarkerFinderTests {
@Test
@DisplayName("finds marker positions inside an attachments section")
void findsMarkerPositions() throws Exception {
byte[] pdfBytes =
pdfWithLines(
"Some intro text",
"Attachments (2)",
"@invoice.pdf (10 KB)",
"@photo.png (4 KB)");
try (PDDocument doc = Loader.loadPDF(pdfBytes)) {
PdfAttachmentHandler.AttachmentMarkerPositionFinder finder =
new PdfAttachmentHandler.AttachmentMarkerPositionFinder();
finder.setSortByPosition(false);
String returned = finder.getText(doc);
// getText is overridden to return an empty string (positions are the payload).
assertEquals("", returned);
List<PdfAttachmentHandler.MarkerPosition> positions = finder.getPositions();
assertEquals(2, positions.size());
List<String> filenames =
positions.stream()
.map(PdfAttachmentHandler.MarkerPosition::getFilename)
.toList();
assertTrue(filenames.contains("invoice.pdf"));
assertTrue(filenames.contains("photo.png"));
for (PdfAttachmentHandler.MarkerPosition p : positions) {
assertEquals("@", p.getCharacter());
assertEquals(0, p.getPageIndex());
}
}
}
@Test
@DisplayName("collects no positions when there is no attachments section")
void noAttachmentSection_noPositions() throws Exception {
byte[] pdfBytes =
pdfWithLines("Plain email body", "Contact us @ support address", "Goodbye");
try (PDDocument doc = Loader.loadPDF(pdfBytes)) {
PdfAttachmentHandler.AttachmentMarkerPositionFinder finder =
new PdfAttachmentHandler.AttachmentMarkerPositionFinder();
finder.getText(doc);
assertTrue(finder.getPositions().isEmpty());
}
}
@Test
@DisplayName("sortByPosition reorders collected positions deterministically")
void sortByPosition_sortsPositions() throws Exception {
byte[] pdfBytes =
pdfWithLines("Attachments (2)", "@first.pdf (1 KB)", "@second.pdf (2 KB)");
try (PDDocument doc = Loader.loadPDF(pdfBytes)) {
PdfAttachmentHandler.AttachmentMarkerPositionFinder finder =
new PdfAttachmentHandler.AttachmentMarkerPositionFinder();
finder.setSortByPosition(true);
finder.getText(doc);
List<PdfAttachmentHandler.MarkerPosition> positions = finder.getPositions();
assertEquals(2, positions.size());
// With descending-Y sorting and same page, the higher-on-page marker comes first.
assertTrue(positions.get(0).getY() >= positions.get(1).getY());
}
}
}
// ----- processInlineImages -------------------------------------------
@Nested
@DisplayName("processInlineImages")
class ProcessInlineImagesTests {
@Test
@DisplayName("replaces a cid: reference with an inline base64 data URI")
void replacesCidWithDataUri() {
byte[] imageData = {(byte) 0x89, 'P', 'N', 'G'};
EmlParser.EmailAttachment img = new EmlParser.EmailAttachment();
img.setEmbedded(true);
img.setContentId("img001");
img.setFilename("pic.png");
img.setContentType("image/png");
img.setData(imageData);
EmlParser.EmailContent content = new EmlParser.EmailContent();
List<EmlParser.EmailAttachment> list = new ArrayList<>();
list.add(img);
content.setAttachments(list);
String html = "<html><body><img src=\"cid:img001\"/></body></html>";
String result = PdfAttachmentHandler.processInlineImages(html, content);
String expectedB64 = Base64.getEncoder().encodeToString(imageData);
assertTrue(result.contains("data:image/png;base64," + expectedB64));
assertFalse(result.contains("cid:img001"));
}
@Test
@DisplayName("leaves a cid: reference untouched when no attachment matches it")
void unmatchedCid_isUnchanged() {
EmlParser.EmailAttachment img = new EmlParser.EmailAttachment();
img.setEmbedded(true);
img.setContentId("known");
img.setFilename("known.png");
img.setContentType("image/png");
img.setData(new byte[] {1, 2, 3});
EmlParser.EmailContent content = new EmlParser.EmailContent();
List<EmlParser.EmailAttachment> list = new ArrayList<>();
list.add(img);
content.setAttachments(list);
String html = "<img src=\"cid:unknown\"/>";
String result = PdfAttachmentHandler.processInlineImages(html, content);
// The unknown cid reference is preserved verbatim.
assertTrue(result.contains("cid:unknown"));
}
@Test
@DisplayName("returns original html when there are no embedded images to map")
void noEmbeddedImages_returnsOriginal() {
EmlParser.EmailAttachment nonEmbedded = new EmlParser.EmailAttachment();
nonEmbedded.setEmbedded(false);
nonEmbedded.setContentId("x");
nonEmbedded.setData(new byte[] {1});
EmlParser.EmailContent content = new EmlParser.EmailContent();
List<EmlParser.EmailAttachment> list = new ArrayList<>();
list.add(nonEmbedded);
content.setAttachments(list);
String html = "<img src=\"cid:x\"/>";
assertEquals(html, PdfAttachmentHandler.processInlineImages(html, content));
}
}
// ----- formatEmailDate (deterministic UTC) ---------------------------
@Nested
@DisplayName("formatEmailDate determinism")
class FormatEmailDateTests {
@Test
@DisplayName("a known instant formats to a stable UTC string regardless of input zone")
void zonedDateTime_formatsToUtc() {
// 2024-06-15 12:00 in Tokyo is 03:00 UTC the same day.
ZonedDateTime tokyo =
ZonedDateTime.of(2024, 6, 15, 12, 0, 0, 0, ZoneId.of("Asia/Tokyo"));
String result = PdfAttachmentHandler.formatEmailDate(tokyo);
assertEquals("Sat, Jun 15, 2024 at 3:00 AM UTC", result);
}
@Test
@DisplayName("Date overload converts a fixed epoch instant to the expected UTC string")
void date_formatsToUtc() {
// Epoch milli 0 == 1970-01-01T00:00:00Z.
String result = PdfAttachmentHandler.formatEmailDate(new Date(0L));
assertEquals("Thu, Jan 1, 1970 at 12:00 AM UTC", result);
}
@Test
@DisplayName("null inputs yield an empty string for both overloads")
void nullInputs_returnEmpty() {
assertEquals("", PdfAttachmentHandler.formatEmailDate((Date) null));
assertEquals("", PdfAttachmentHandler.formatEmailDate((ZonedDateTime) null));
}
}
}
@@ -0,0 +1,551 @@
package stirling.software.common.util;
import static org.junit.jupiter.api.Assertions.*;
import static org.mockito.Mockito.*;
import java.awt.Color;
import java.awt.Graphics2D;
import java.awt.image.BufferedImage;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import javax.imageio.ImageIO;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDPage;
import org.apache.pdfbox.pdmodel.PDPageContentStream;
import org.apache.pdfbox.pdmodel.common.PDRectangle;
import org.apache.pdfbox.pdmodel.font.PDType1Font;
import org.apache.pdfbox.pdmodel.font.Standard14Fonts;
import org.apache.pdfbox.pdmodel.graphics.image.PDImageXObject;
import org.apache.pdfbox.rendering.ImageType;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import org.springframework.http.MediaType;
import org.springframework.mock.web.MockMultipartFile;
import org.springframework.web.multipart.MultipartFile;
import stirling.software.common.service.CustomPDFDocumentFactory;
/**
* Additional unit tests for {@link PdfUtils} targeting methods not exercised by {@code
* PdfUtilsTest}: convertFromPdf, convertPdfToPdfImage, imageToPdf, addImageToDocument,
* overlayImage, containsTextInFile and the error branch of pageSize.
*/
@ExtendWith(MockitoExtension.class)
class PdfUtilsGapTest {
@Mock private CustomPDFDocumentFactory pdfDocumentFactory;
// ---- helpers ------------------------------------------------------------
/** Builds a tiny single-page PDF and returns it serialized to bytes. */
private static byte[] simplePdfBytes() throws IOException {
try (PDDocument doc = new PDDocument()) {
doc.addPage(new PDPage(PDRectangle.A4));
ByteArrayOutputStream baos = new ByteArrayOutputStream();
doc.save(baos);
return baos.toByteArray();
}
}
/** Builds a PDF with the given number of empty A4 pages. */
private static PDDocument docWithPages(int pages) {
PDDocument doc = new PDDocument();
for (int i = 0; i < pages; i++) {
doc.addPage(new PDPage(PDRectangle.A4));
}
return doc;
}
/**
* Builds a PDF with the given number of tiny pages. convertPdfToPdfImage rasterises every page
* at 300 DPI, so page area drives the cost; tiny pages keep render work minimal while still
* exercising the per-page loop. Page size is non-square so dimension preservation stays
* verifiable.
*/
private static PDDocument docWithTinyPages(int pages, float width, float height) {
PDDocument doc = new PDDocument();
for (int i = 0; i < pages; i++) {
doc.addPage(new PDPage(new PDRectangle(width, height)));
}
return doc;
}
/** Builds a PDF whose pages each contain the given text phrase. */
private static PDDocument docWithText(String... pageTexts) throws IOException {
PDDocument doc = new PDDocument();
for (String text : pageTexts) {
PDPage page = new PDPage(PDRectangle.A4);
doc.addPage(page);
try (PDPageContentStream cs = new PDPageContentStream(doc, page)) {
cs.beginText();
cs.setFont(new PDType1Font(Standard14Fonts.FontName.HELVETICA), 12);
cs.newLineAtOffset(100, 700);
cs.showText(text);
cs.endText();
}
}
return doc;
}
/** Encodes a small solid-color image to bytes in the requested format. */
private static byte[] imageBytes(String format, Color color) throws IOException {
BufferedImage img = new BufferedImage(20, 20, BufferedImage.TYPE_INT_RGB);
Graphics2D g = img.createGraphics();
g.setColor(color);
g.fillRect(0, 0, 20, 20);
g.dispose();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ImageIO.write(img, format, baos);
return baos.toByteArray();
}
// ---- convertFromPdf -----------------------------------------------------
@Nested
@DisplayName("convertFromPdf")
class ConvertFromPdf {
@Test
@DisplayName("single PNG image is produced from a one-page PDF")
void singlePng() throws Exception {
byte[] bytes = simplePdfBytes();
when(pdfDocumentFactory.load(bytes)).thenReturn(docWithPages(1));
byte[] out =
PdfUtils.convertFromPdf(
pdfDocumentFactory, bytes, "png", ImageType.RGB, true, 72, "doc", true);
assertNotNull(out);
assertTrue(out.length > 0);
// A valid PNG starts with the 8-byte PNG signature.
assertEquals((byte) 0x89, out[0]);
assertEquals('P', out[1]);
assertEquals('N', out[2]);
assertEquals('G', out[3]);
}
@Test
@DisplayName("single combined JPEG image is produced for multi-page PDF")
void singleJpegMultiPage() throws Exception {
byte[] bytes = simplePdfBytes();
when(pdfDocumentFactory.load(bytes)).thenReturn(docWithPages(2));
byte[] out =
PdfUtils.convertFromPdf(
pdfDocumentFactory,
bytes,
"jpg",
ImageType.RGB,
true,
72,
"doc",
false);
assertNotNull(out);
assertTrue(out.length > 0);
// JPEG magic bytes.
assertEquals((byte) 0xFF, out[0]);
assertEquals((byte) 0xD8, out[1]);
}
@Test
@DisplayName("single TIFF image sequence is produced for multi-page PDF")
void singleTiffMultiPage() throws Exception {
byte[] bytes = simplePdfBytes();
when(pdfDocumentFactory.load(bytes)).thenReturn(docWithPages(2));
byte[] out =
PdfUtils.convertFromPdf(
pdfDocumentFactory,
bytes,
"tiff",
ImageType.RGB,
true,
72,
"doc",
true);
assertNotNull(out);
assertTrue(out.length > 0);
}
@Test
@DisplayName("non-single image mode returns a non-empty zip of per-page images")
void zipOfImages() throws Exception {
byte[] bytes = simplePdfBytes();
when(pdfDocumentFactory.load(bytes)).thenReturn(docWithPages(2));
byte[] out =
PdfUtils.convertFromPdf(
pdfDocumentFactory,
bytes,
"png",
ImageType.RGB,
false,
72,
"myfile",
true);
assertNotNull(out);
assertTrue(out.length > 0);
// ZIP local-file-header magic "PK\003\004".
assertEquals('P', out[0]);
assertEquals('K', out[1]);
}
@Test
@DisplayName("DPI above the safe limit throws IllegalArgumentException")
void dpiTooHighThrows() {
byte[] bytes = new byte[] {1, 2, 3};
// The DPI check happens before the document is loaded.
assertThrows(
IllegalArgumentException.class,
() ->
PdfUtils.convertFromPdf(
pdfDocumentFactory,
bytes,
"png",
ImageType.RGB,
true,
9999,
"doc",
true));
}
@Test
@DisplayName("annotations excluded path still renders successfully")
void withoutAnnotations() throws Exception {
byte[] bytes = simplePdfBytes();
when(pdfDocumentFactory.load(bytes)).thenReturn(docWithPages(1));
byte[] out =
PdfUtils.convertFromPdf(
pdfDocumentFactory,
bytes,
"png",
ImageType.RGB,
true,
72,
"doc",
false);
assertNotNull(out);
assertTrue(out.length > 0);
}
}
// ---- convertPdfToPdfImage -----------------------------------------------
@Nested
@DisplayName("convertPdfToPdfImage")
class ConvertPdfToPdfImage {
@Test
@DisplayName("returns a new document with the same page count")
void preservesPageCount() throws IOException {
// Page size is irrelevant to the count assertion; tiny pages avoid a 300 DPI A4 raster.
try (PDDocument source = docWithTinyPages(2, 6f, 9f);
PDDocument result = PdfUtils.convertPdfToPdfImage(source)) {
assertNotNull(result);
assertEquals(2, result.getNumberOfPages());
}
}
@Test
@DisplayName("preserves page dimensions of the source")
void preservesPageSize() throws IOException {
// A small non-square page still proves width/height are carried through (and not
// swapped) without rastering a full LETTER page at 300 DPI.
float width = 60f;
float height = 90f;
try (PDDocument source = new PDDocument()) {
source.addPage(new PDPage(new PDRectangle(width, height)));
try (PDDocument result = PdfUtils.convertPdfToPdfImage(source)) {
PDRectangle box = result.getPage(0).getMediaBox();
assertEquals(width, box.getWidth(), 0.5f);
assertEquals(height, box.getHeight(), 0.5f);
}
}
}
@Test
@DisplayName("empty document yields an empty document")
void emptyDocument() throws IOException {
try (PDDocument source = new PDDocument();
PDDocument result = PdfUtils.convertPdfToPdfImage(source)) {
assertEquals(0, result.getNumberOfPages());
}
}
}
// ---- imageToPdf ---------------------------------------------------------
@Nested
@DisplayName("imageToPdf")
class ImageToPdf {
private byte[] runImageToPdf(MultipartFile[] files, String fitOption, boolean autoRotate)
throws IOException {
when(pdfDocumentFactory.createNewDocument()).thenReturn(new PDDocument());
return PdfUtils.imageToPdf(files, fitOption, autoRotate, "color", pdfDocumentFactory);
}
@Test
@DisplayName("single PNG image becomes a one-page PDF")
void singlePngImage() throws IOException {
MockMultipartFile file =
new MockMultipartFile(
"file",
"image.png",
MediaType.IMAGE_PNG_VALUE,
imageBytes("png", Color.RED));
byte[] pdfOut = runImageToPdf(new MultipartFile[] {file}, "fillPage", false);
assertNotNull(pdfOut);
try (PDDocument doc = org.apache.pdfbox.Loader.loadPDF(pdfOut)) {
assertEquals(1, doc.getNumberOfPages());
}
}
@Test
@DisplayName("JPEG image uses the lossy factory path and produces a PDF")
void jpegImage() throws IOException {
MockMultipartFile file =
new MockMultipartFile(
"file",
"image.jpg",
MediaType.IMAGE_JPEG_VALUE,
imageBytes("jpg", Color.BLUE));
byte[] pdfOut = runImageToPdf(new MultipartFile[] {file}, "maintainAspectRatio", false);
assertNotNull(pdfOut);
try (PDDocument doc = org.apache.pdfbox.Loader.loadPDF(pdfOut)) {
assertEquals(1, doc.getNumberOfPages());
}
}
@Test
@DisplayName("fitDocumentToImage sizes the page to the image")
void fitDocumentToImage() throws IOException {
MockMultipartFile file =
new MockMultipartFile(
"file",
"image.png",
MediaType.IMAGE_PNG_VALUE,
imageBytes("png", Color.GREEN));
byte[] pdfOut = runImageToPdf(new MultipartFile[] {file}, "fitDocumentToImage", false);
try (PDDocument doc = org.apache.pdfbox.Loader.loadPDF(pdfOut)) {
PDRectangle box = doc.getPage(0).getMediaBox();
assertEquals(20f, box.getWidth(), 0.5f);
assertEquals(20f, box.getHeight(), 0.5f);
}
}
@Test
@DisplayName("multiple images become multiple pages")
void multipleImages() throws IOException {
MockMultipartFile a =
new MockMultipartFile(
"file",
"a.png",
MediaType.IMAGE_PNG_VALUE,
imageBytes("png", Color.RED));
MockMultipartFile b =
new MockMultipartFile(
"file",
"b.png",
MediaType.IMAGE_PNG_VALUE,
imageBytes("png", Color.BLUE));
byte[] pdfOut = runImageToPdf(new MultipartFile[] {a, b}, "fillPage", true);
try (PDDocument doc = org.apache.pdfbox.Loader.loadPDF(pdfOut)) {
assertEquals(2, doc.getNumberOfPages());
}
}
}
// ---- addImageToDocument -------------------------------------------------
@Nested
@DisplayName("addImageToDocument")
class AddImageToDocument {
private PDImageXObject portraitImage(PDDocument doc) throws IOException {
BufferedImage img = new BufferedImage(40, 80, BufferedImage.TYPE_INT_RGB);
return org.apache.pdfbox.pdmodel.graphics.image.LosslessFactory.createFromImage(
doc, img);
}
private PDImageXObject landscapeImage(PDDocument doc) throws IOException {
BufferedImage img = new BufferedImage(80, 40, BufferedImage.TYPE_INT_RGB);
return org.apache.pdfbox.pdmodel.graphics.image.LosslessFactory.createFromImage(
doc, img);
}
@Test
@DisplayName("fillPage adds an A4 page")
void fillPage() throws IOException {
try (PDDocument doc = new PDDocument()) {
PdfUtils.addImageToDocument(doc, portraitImage(doc), "fillPage", false);
assertEquals(1, doc.getNumberOfPages());
PDRectangle box = doc.getPage(0).getMediaBox();
assertEquals(PDRectangle.A4.getWidth(), box.getWidth(), 0.5f);
}
}
@Test
@DisplayName("maintainAspectRatio adds an A4 page and centers the image")
void maintainAspectRatio() throws IOException {
try (PDDocument doc = new PDDocument()) {
PdfUtils.addImageToDocument(doc, portraitImage(doc), "maintainAspectRatio", false);
assertEquals(1, doc.getNumberOfPages());
}
}
@Test
@DisplayName("fitDocumentToImage sizes the page to the image bounds")
void fitDocumentToImage() throws IOException {
try (PDDocument doc = new PDDocument()) {
PdfUtils.addImageToDocument(doc, portraitImage(doc), "fitDocumentToImage", false);
PDRectangle box = doc.getPage(0).getMediaBox();
assertEquals(40f, box.getWidth(), 0.5f);
assertEquals(80f, box.getHeight(), 0.5f);
}
}
@Test
@DisplayName("autoRotate with a landscape image swaps to landscape A4")
void autoRotateLandscape() throws IOException {
try (PDDocument doc = new PDDocument()) {
PdfUtils.addImageToDocument(doc, landscapeImage(doc), "maintainAspectRatio", true);
PDRectangle box = doc.getPage(0).getMediaBox();
// Landscape: width should now exceed height.
assertTrue(box.getWidth() > box.getHeight());
}
}
@Test
@DisplayName("unknown fit option still adds a page without drawing")
void unknownFitOption() throws IOException {
try (PDDocument doc = new PDDocument()) {
PdfUtils.addImageToDocument(doc, portraitImage(doc), "unknownOption", false);
assertEquals(1, doc.getNumberOfPages());
}
}
}
// ---- overlayImage -------------------------------------------------------
@Nested
@DisplayName("overlayImage")
class OverlayImage {
@Test
@DisplayName("overlays only the first page when everyPage is false")
void firstPageOnly() throws IOException {
byte[] pdf = simplePdfBytes();
when(pdfDocumentFactory.load(pdf)).thenReturn(docWithPages(3));
byte[] image = imageBytes("png", Color.RED);
byte[] out = PdfUtils.overlayImage(pdfDocumentFactory, pdf, image, 10f, 10f, false);
assertNotNull(out);
try (PDDocument doc = org.apache.pdfbox.Loader.loadPDF(out)) {
assertEquals(3, doc.getNumberOfPages());
}
}
@Test
@DisplayName("overlays every page when everyPage is true")
void everyPage() throws IOException {
byte[] pdf = simplePdfBytes();
when(pdfDocumentFactory.load(pdf)).thenReturn(docWithPages(2));
byte[] image = imageBytes("png", Color.BLUE);
byte[] out = PdfUtils.overlayImage(pdfDocumentFactory, pdf, image, 0f, 0f, true);
assertNotNull(out);
assertTrue(out.length > 0);
try (PDDocument doc = org.apache.pdfbox.Loader.loadPDF(out)) {
assertEquals(2, doc.getNumberOfPages());
}
}
}
// ---- containsTextInFile -------------------------------------------------
@Nested
@DisplayName("containsTextInFile")
class ContainsTextInFile {
@Test
@DisplayName("finds text when searching all pages")
void allPagesMatch() throws IOException {
PDDocument doc = docWithText("HelloWorld");
assertTrue(PdfUtils.containsTextInFile(doc, "HelloWorld", "all"));
}
@Test
@DisplayName("null pagesToCheck is treated as all pages")
void nullPagesTreatedAsAll() throws IOException {
PDDocument doc = docWithText("FindThis");
assertTrue(PdfUtils.containsTextInFile(doc, "FindThis", null));
}
@Test
@DisplayName("returns false when text is absent")
void noMatch() throws IOException {
PDDocument doc = docWithText("SomeText");
assertFalse(PdfUtils.containsTextInFile(doc, "Missing", "all"));
}
@Test
@DisplayName("matches text on an individual page number")
void individualPage() throws IOException {
PDDocument doc = docWithText("PageOne", "PageTwo");
assertTrue(PdfUtils.containsTextInFile(doc, "PageTwo", "2"));
}
@Test
@DisplayName("matches text within a page range")
void pageRange() throws IOException {
PDDocument doc = docWithText("Alpha", "Beta", "Gamma");
assertTrue(PdfUtils.containsTextInFile(doc, "Gamma", "1-3"));
}
@Test
@DisplayName("whitespace in the page spec is stripped before parsing")
void whitespaceStripped() throws IOException {
PDDocument doc = docWithText("One", "Two");
assertTrue(PdfUtils.containsTextInFile(doc, "Two", " 1 , 2 "));
}
}
// ---- pageSize error branch ---------------------------------------------
@Nested
@DisplayName("pageSize parsing")
class PageSizeParsing {
@Test
@DisplayName("non-numeric expected size throws NumberFormatException")
void nonNumericThrows() throws IOException {
try (PDDocument doc = new PDDocument()) {
doc.addPage(new PDPage(PDRectangle.A4));
assertThrows(
NumberFormatException.class, () -> PdfUtils.pageSize(doc, "widthxheight"));
}
}
}
}
@@ -0,0 +1,634 @@
package stirling.software.common.util;
import static org.junit.jupiter.api.Assertions.*;
import java.io.IOException;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.ArrayList;
import java.util.List;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import stirling.software.common.model.ApplicationProperties;
/**
* Gap-filling unit tests for {@link ProcessExecutor}. Focused on the pure logic that can be
* exercised without launching any real OS process: command validation branches, the unoserver
* endpoint helper methods (via reflection), the {@link ProcessExecutor.Processes} enum, the
* singleton/getInstance behaviour, the static unoserver pool setter, and the nested {@link
* ProcessExecutor.ProcessExecutorResult} value type.
*/
class ProcessExecutorGapTest {
// ----- reflection helpers -------------------------------------------------
private void invokeValidateCommand(ProcessExecutor executor, List<String> command)
throws Exception {
Method method = ProcessExecutor.class.getDeclaredMethod("validateCommand", List.class);
method.setAccessible(true);
try {
method.invoke(executor, command);
} catch (InvocationTargetException e) {
throw (Exception) e.getCause();
}
}
@SuppressWarnings("unchecked")
private List<String> invokeStripUnoEndpointArgs(ProcessExecutor executor, List<String> command)
throws Exception {
Method method = ProcessExecutor.class.getDeclaredMethod("stripUnoEndpointArgs", List.class);
method.setAccessible(true);
return (List<String>) method.invoke(executor, command);
}
@SuppressWarnings("unchecked")
private List<String> invokeApplyUnoServerEndpoint(
ProcessExecutor executor,
List<String> command,
ApplicationProperties.ProcessExecutor.UnoServerEndpoint endpoint)
throws Exception {
Method method =
ProcessExecutor.class.getDeclaredMethod(
"applyUnoServerEndpoint",
List.class,
ApplicationProperties.ProcessExecutor.UnoServerEndpoint.class);
method.setAccessible(true);
return (List<String>) method.invoke(executor, command, endpoint);
}
private boolean invokeShouldUseUnoServerPool(ProcessExecutor executor, List<String> command)
throws Exception {
Method method =
ProcessExecutor.class.getDeclaredMethod("shouldUseUnoServerPool", List.class);
method.setAccessible(true);
return (boolean) method.invoke(executor, command);
}
private ProcessExecutor qpdfExecutor() {
return ProcessExecutor.getInstance(ProcessExecutor.Processes.QPDF);
}
private ProcessExecutor libreOfficeExecutor() {
return ProcessExecutor.getInstance(ProcessExecutor.Processes.LIBRE_OFFICE);
}
/** The static unoserver pool is global state; clear it after every test that touches it. */
@AfterEach
void resetUnoServerPool() {
ProcessExecutor.setUnoServerPool(null);
}
// ----- validateCommand deeper branches -----------------------------------
@Nested
@DisplayName("validateCommand path/executable branches")
class ValidateCommandPathTests {
@Test
@DisplayName("absolute path executable that does not exist is rejected")
void absolutePathExecutableMissing() {
String bogus =
System.getProperty("os.name").toLowerCase().contains("win")
? "C:\\definitely\\does\\not\\exist\\tool.exe"
: "/definitely/does/not/exist/tool";
IllegalArgumentException ex =
assertThrows(
IllegalArgumentException.class,
() -> invokeValidateCommand(qpdfExecutor(), List.of(bogus)));
assertTrue(ex.getMessage().contains("does not exist"));
}
@Test
@DisplayName("path that exists but is a directory is rejected (not a regular file)")
void directoryPathExecutableRejected(@TempDir Path tempDir) {
String dirPath = tempDir.toString();
// Ensure the path contains a separator so the path-based validation branch is taken.
assertTrue(dirPath.contains("/") || dirPath.contains("\\"));
IllegalArgumentException ex =
assertThrows(
IllegalArgumentException.class,
() -> invokeValidateCommand(qpdfExecutor(), List.of(dirPath)));
assertTrue(ex.getMessage().contains("not a regular file"));
}
@Test
@DisplayName("absolute path to an existing regular file passes validation")
void existingRegularFileExecutablePasses(@TempDir Path tempDir) throws Exception {
Path file = tempDir.resolve("fakebinary");
Files.writeString(file, "#!/bin/sh\n");
assertTrue(Files.exists(file));
// Should not throw.
invokeValidateCommand(qpdfExecutor(), List.of(file.toString(), "--version"));
}
@Test
@DisplayName("path traversal anywhere in the executable is rejected")
void pathTraversalInExecutable() {
IllegalArgumentException ex =
assertThrows(
IllegalArgumentException.class,
() ->
invokeValidateCommand(
qpdfExecutor(), List.of("/usr/bin/../bin/tool")));
assertTrue(ex.getMessage().contains("path traversal"));
}
@Test
@DisplayName("null byte / newline checks run across every argument, not just the first")
void invalidCharactersInLaterArgument() {
assertThrows(
IllegalArgumentException.class,
() -> invokeValidateCommand(qpdfExecutor(), List.of("qpdf", "ok", "bad\0arg")));
assertThrows(
IllegalArgumentException.class,
() -> invokeValidateCommand(qpdfExecutor(), List.of("qpdf", "ok", "bad\narg")));
assertThrows(
IllegalArgumentException.class,
() -> invokeValidateCommand(qpdfExecutor(), List.of("qpdf", "ok", "bad\rarg")));
}
@Test
@DisplayName("relative simple command (no separators) is trusted and passes")
void relativeSimpleCommandPasses() throws Exception {
invokeValidateCommand(qpdfExecutor(), List.of("qpdf", "--help"));
}
@Test
@DisplayName("null first-argument executable is rejected")
void nullExecutableRejected() {
List<String> command = new ArrayList<>();
command.add(null);
// null arg is caught by the per-arg null check before the executable check.
assertThrows(
IllegalArgumentException.class,
() -> invokeValidateCommand(qpdfExecutor(), command));
}
}
// ----- stripUnoEndpointArgs ----------------------------------------------
@Nested
@DisplayName("stripUnoEndpointArgs")
class StripUnoEndpointArgsTests {
@Test
@DisplayName("removes space-separated --host/--port/--host-location/--protocol pairs")
void stripsSpaceSeparatedArgs() throws Exception {
List<String> input =
List.of(
"unoconvert",
"--host",
"1.2.3.4",
"--port",
"9999",
"--host-location",
"remote",
"--protocol",
"https",
"in.docx",
"out.pdf");
List<String> result = invokeStripUnoEndpointArgs(qpdfExecutor(), input);
assertEquals(List.of("unoconvert", "in.docx", "out.pdf"), result);
}
@Test
@DisplayName("removes equals-form --host=.../--port=... arguments")
void stripsEqualsFormArgs() throws Exception {
List<String> input =
List.of(
"unoconvert",
"--host=5.6.7.8",
"--port=4002",
"--host-location=local",
"--protocol=http",
"doc.odt");
List<String> result = invokeStripUnoEndpointArgs(qpdfExecutor(), input);
assertEquals(List.of("unoconvert", "doc.odt"), result);
}
@Test
@DisplayName("leaves a command without endpoint args unchanged")
void leavesPlainCommandUntouched() throws Exception {
List<String> input = List.of("unoconvert", "in.docx", "out.pdf");
List<String> result = invokeStripUnoEndpointArgs(qpdfExecutor(), input);
assertEquals(input, result);
}
@Test
@DisplayName("returns a fresh list, not the same instance")
void returnsNewList() throws Exception {
List<String> input = new ArrayList<>(List.of("unoconvert", "a", "b"));
List<String> result = invokeStripUnoEndpointArgs(qpdfExecutor(), input);
assertNotSame(input, result);
}
}
// ----- applyUnoServerEndpoint --------------------------------------------
private ApplicationProperties.ProcessExecutor.UnoServerEndpoint endpoint(
String host, int port, String hostLocation, String protocol) {
ApplicationProperties.ProcessExecutor.UnoServerEndpoint ep =
new ApplicationProperties.ProcessExecutor.UnoServerEndpoint();
ep.setHost(host);
ep.setPort(port);
ep.setHostLocation(hostLocation);
ep.setProtocol(protocol);
return ep;
}
@Nested
@DisplayName("applyUnoServerEndpoint")
class ApplyUnoServerEndpointTests {
@Test
@DisplayName(
"injects --host/--port after the executable, defaults omit host-location and protocol")
void injectsHostAndPortWithDefaults() throws Exception {
List<String> command = List.of("unoconvert", "in.docx", "out.pdf");
ApplicationProperties.ProcessExecutor.UnoServerEndpoint ep =
endpoint("9.9.9.9", 7777, "auto", "http");
List<String> result = invokeApplyUnoServerEndpoint(qpdfExecutor(), command, ep);
assertEquals(
List.of(
"unoconvert",
"--host",
"9.9.9.9",
"--port",
"7777",
"in.docx",
"out.pdf"),
result);
}
@Test
@DisplayName("non-default host-location and protocol are injected")
void injectsHostLocationAndProtocolWhenNonDefault() throws Exception {
List<String> command = List.of("unoconvert", "in.docx");
ApplicationProperties.ProcessExecutor.UnoServerEndpoint ep =
endpoint("10.0.0.5", 2200, "remote", "https");
List<String> result = invokeApplyUnoServerEndpoint(qpdfExecutor(), command, ep);
assertEquals(
List.of(
"unoconvert",
"--host",
"10.0.0.5",
"--port",
"2200",
"--host-location",
"remote",
"--protocol",
"https",
"in.docx"),
result);
}
@Test
@DisplayName("blank host falls back to 127.0.0.1 and non-positive port falls back to 2003")
void appliesHostAndPortFallbacks() throws Exception {
List<String> command = List.of("unoconvert", "in.docx");
ApplicationProperties.ProcessExecutor.UnoServerEndpoint ep =
endpoint(" ", 0, "auto", "http");
List<String> result = invokeApplyUnoServerEndpoint(qpdfExecutor(), command, ep);
assertEquals(
List.of("unoconvert", "--host", "127.0.0.1", "--port", "2003", "in.docx"),
result);
}
@Test
@DisplayName("invalid host-location and protocol values are normalised to defaults")
void invalidHostLocationAndProtocolNormalised() throws Exception {
List<String> command = List.of("unoconvert", "in.docx");
ApplicationProperties.ProcessExecutor.UnoServerEndpoint ep =
endpoint("1.1.1.1", 3000, "sideways", "gopher");
List<String> result = invokeApplyUnoServerEndpoint(qpdfExecutor(), command, ep);
// Both invalid -> normalised to defaults (auto/http) -> neither injected.
assertEquals(
List.of("unoconvert", "--host", "1.1.1.1", "--port", "3000", "in.docx"),
result);
}
@Test
@DisplayName("host-location and protocol matching is case-insensitive and trimmed")
void hostLocationAndProtocolCaseInsensitive() throws Exception {
List<String> command = List.of("unoconvert", "in.docx");
ApplicationProperties.ProcessExecutor.UnoServerEndpoint ep =
endpoint("1.1.1.1", 3000, " REMOTE ", " HTTPS ");
List<String> result = invokeApplyUnoServerEndpoint(qpdfExecutor(), command, ep);
assertEquals(
List.of(
"unoconvert",
"--host",
"1.1.1.1",
"--port",
"3000",
"--host-location",
"remote",
"--protocol",
"https",
"in.docx"),
result);
}
@Test
@DisplayName("existing endpoint args are stripped before re-injection")
void stripsExistingEndpointArgsBeforeInjecting() throws Exception {
List<String> command = List.of("unoconvert", "--host", "old", "--port", "1", "in.docx");
ApplicationProperties.ProcessExecutor.UnoServerEndpoint ep =
endpoint("2.2.2.2", 2222, "auto", "http");
List<String> result = invokeApplyUnoServerEndpoint(qpdfExecutor(), command, ep);
assertEquals(
List.of("unoconvert", "--host", "2.2.2.2", "--port", "2222", "in.docx"),
result);
}
@Test
@DisplayName("null endpoint returns the command unchanged")
void nullEndpointReturnsCommandUnchanged() throws Exception {
List<String> command = List.of("unoconvert", "in.docx");
List<String> result = invokeApplyUnoServerEndpoint(qpdfExecutor(), command, null);
assertEquals(command, result);
}
@Test
@DisplayName("empty command returns the command unchanged")
void emptyCommandReturnedUnchanged() throws Exception {
List<String> command = List.of();
ApplicationProperties.ProcessExecutor.UnoServerEndpoint ep =
endpoint("1.1.1.1", 2003, "auto", "http");
List<String> result = invokeApplyUnoServerEndpoint(qpdfExecutor(), command, ep);
assertEquals(command, result);
}
}
// ----- shouldUseUnoServerPool --------------------------------------------
@Nested
@DisplayName("shouldUseUnoServerPool")
class ShouldUseUnoServerPoolTests {
private UnoServerPool nonEmptyPool() {
ApplicationProperties.ProcessExecutor.UnoServerEndpoint ep =
new ApplicationProperties.ProcessExecutor.UnoServerEndpoint();
return new UnoServerPool(List.of(ep));
}
@Test
@DisplayName(
"false for non-LIBRE_OFFICE process type even with a pool and unoconvert command")
void falseForNonLibreOfficeProcessType() throws Exception {
ProcessExecutor.setUnoServerPool(nonEmptyPool());
assertFalse(
invokeShouldUseUnoServerPool(qpdfExecutor(), List.of("unoconvert", "in.docx")));
}
@Test
@DisplayName("false when no pool is configured")
void falseWhenPoolNull() throws Exception {
ProcessExecutor.setUnoServerPool(null);
assertFalse(
invokeShouldUseUnoServerPool(
libreOfficeExecutor(), List.of("unoconvert", "in.docx")));
}
@Test
@DisplayName("false when the configured pool is empty")
void falseWhenPoolEmpty() throws Exception {
ProcessExecutor.setUnoServerPool(new UnoServerPool(List.of()));
assertFalse(
invokeShouldUseUnoServerPool(
libreOfficeExecutor(), List.of("unoconvert", "in.docx")));
}
@Test
@DisplayName("false for null or empty command")
void falseForNullOrEmptyCommand() throws Exception {
ProcessExecutor.setUnoServerPool(nonEmptyPool());
assertFalse(invokeShouldUseUnoServerPool(libreOfficeExecutor(), null));
assertFalse(invokeShouldUseUnoServerPool(libreOfficeExecutor(), List.of()));
}
@Test
@DisplayName("true for a plain unoconvert command with a non-empty pool")
void trueForUnoconvertCommand() throws Exception {
ProcessExecutor.setUnoServerPool(nonEmptyPool());
assertTrue(
invokeShouldUseUnoServerPool(
libreOfficeExecutor(), List.of("unoconvert", "in.docx", "out.pdf")));
}
@Test
@DisplayName("true for a unoconvert path with directories and a .exe extension")
void trueForUnoconvertWithPathAndExeExtension() throws Exception {
ProcessExecutor.setUnoServerPool(nonEmptyPool());
assertTrue(
invokeShouldUseUnoServerPool(
libreOfficeExecutor(),
List.of("C:\\tools\\bin\\unoconvert.exe", "in.docx")));
assertTrue(
invokeShouldUseUnoServerPool(
libreOfficeExecutor(),
List.of("/usr/local/bin/unoconvert", "in.docx")));
}
@Test
@DisplayName("true for the legacy 'unoconv' executable name")
void trueForLegacyUnoconv() throws Exception {
ProcessExecutor.setUnoServerPool(nonEmptyPool());
assertTrue(
invokeShouldUseUnoServerPool(
libreOfficeExecutor(), List.of("unoconv", "in.docx")));
}
@Test
@DisplayName("false for soffice, which must not be routed through the pool")
void falseForSoffice() throws Exception {
ProcessExecutor.setUnoServerPool(nonEmptyPool());
assertFalse(
invokeShouldUseUnoServerPool(
libreOfficeExecutor(),
List.of("/usr/bin/soffice", "--headless", "in.docx")));
}
}
// ----- Processes enum -----------------------------------------------------
@Nested
@DisplayName("Processes enum")
class ProcessesEnumTests {
@Test
@DisplayName("contains all expected process types")
void containsExpectedValues() {
ProcessExecutor.Processes[] values = ProcessExecutor.Processes.values();
assertEquals(13, values.length);
assertEquals(
ProcessExecutor.Processes.LIBRE_OFFICE,
ProcessExecutor.Processes.valueOf("LIBRE_OFFICE"));
assertEquals(
ProcessExecutor.Processes.CFF_CONVERTER,
ProcessExecutor.Processes.valueOf("CFF_CONVERTER"));
assertEquals(
ProcessExecutor.Processes.FFMPEG, ProcessExecutor.Processes.valueOf("FFMPEG"));
}
@Test
@DisplayName("valueOf rejects an unknown name")
void valueOfRejectsUnknown() {
assertThrows(
IllegalArgumentException.class,
() -> ProcessExecutor.Processes.valueOf("NOT_A_PROCESS"));
}
@Test
@DisplayName("getInstance resolves a non-null singleton for every enum value")
void getInstanceForEveryProcessType() {
for (ProcessExecutor.Processes p : ProcessExecutor.Processes.values()) {
ProcessExecutor instance = ProcessExecutor.getInstance(p);
assertNotNull(instance, "instance should not be null for " + p);
// Same key returns the cached singleton.
assertSame(instance, ProcessExecutor.getInstance(p));
}
}
}
// ----- getInstance / liveUpdates -----------------------------------------
@Nested
@DisplayName("getInstance behaviour")
class GetInstanceTests {
@Test
@DisplayName("single-arg getInstance delegates to liveUpdates=true and is cached")
void singleArgDelegatesAndCaches() {
ProcessExecutor a = ProcessExecutor.getInstance(ProcessExecutor.Processes.GHOSTSCRIPT);
ProcessExecutor b =
ProcessExecutor.getInstance(ProcessExecutor.Processes.GHOSTSCRIPT, true);
assertSame(a, b);
}
@Test
@DisplayName("the liveUpdates flag of the first call wins because the instance is cached")
void firstCallWinsForCachedInstance() {
// First resolution for this type fixes its configuration.
ProcessExecutor first =
ProcessExecutor.getInstance(ProcessExecutor.Processes.OCR_MY_PDF, false);
ProcessExecutor second =
ProcessExecutor.getInstance(ProcessExecutor.Processes.OCR_MY_PDF, true);
assertSame(first, second);
}
}
// ----- setUnoServerPool ---------------------------------------------------
@Nested
@DisplayName("setUnoServerPool")
class SetUnoServerPoolTests {
@Test
@DisplayName("setting then clearing the pool flips shouldUseUnoServerPool")
void poolSetterAffectsRouting() throws Exception {
ProcessExecutor exec = libreOfficeExecutor();
List<String> command = List.of("unoconvert", "in.docx");
ProcessExecutor.setUnoServerPool(null);
assertFalse(invokeShouldUseUnoServerPool(exec, command));
ApplicationProperties.ProcessExecutor.UnoServerEndpoint ep =
new ApplicationProperties.ProcessExecutor.UnoServerEndpoint();
ProcessExecutor.setUnoServerPool(new UnoServerPool(List.of(ep)));
assertTrue(invokeShouldUseUnoServerPool(exec, command));
ProcessExecutor.setUnoServerPool(null);
assertFalse(invokeShouldUseUnoServerPool(exec, command));
}
}
// ----- ProcessExecutorResult ---------------------------------------------
@Nested
@DisplayName("ProcessExecutorResult value type")
class ProcessExecutorResultTests {
@Test
@DisplayName("constructor stores rc and messages; setters mutate them")
void constructorAndSetters() {
ProcessExecutor exec = qpdfExecutor();
ProcessExecutor.ProcessExecutorResult result = exec.new ProcessExecutorResult(0, "ok");
assertEquals(0, result.getRc());
assertEquals("ok", result.getMessages());
result.setRc(42);
result.setMessages("boom");
assertEquals(42, result.getRc());
assertEquals("boom", result.getMessages());
}
@Test
@DisplayName("messages may be null")
void allowsNullMessages() {
ProcessExecutor exec = qpdfExecutor();
ProcessExecutor.ProcessExecutorResult result = exec.new ProcessExecutorResult(3, null);
assertEquals(3, result.getRc());
assertNull(result.getMessages());
}
}
// ----- runCommandWithOutputHandling validation entry point ---------------
@Nested
@DisplayName("runCommandWithOutputHandling validation (no process launched)")
class RunCommandValidationTests {
@Test
@DisplayName("empty command is rejected before any process is started")
void emptyCommandRejected() {
ProcessExecutor exec = qpdfExecutor();
assertThrows(
IllegalArgumentException.class,
() -> exec.runCommandWithOutputHandling(List.of()));
}
@Test
@DisplayName("command containing a null byte is rejected before any process is started")
void nullByteCommandRejected() {
ProcessExecutor exec = qpdfExecutor();
assertThrows(
IllegalArgumentException.class,
() -> exec.runCommandWithOutputHandling(List.of("qpdf", "bad\0arg")));
}
@Test
@DisplayName("absolute non-existent executable is rejected before any process is started")
void missingAbsoluteExecutableRejected() {
ProcessExecutor exec = qpdfExecutor();
String bogus =
System.getProperty("os.name").toLowerCase().contains("win")
? "C:\\no\\such\\tool.exe"
: "/no/such/tool";
IllegalArgumentException ex =
assertThrows(
IllegalArgumentException.class,
() -> exec.runCommandWithOutputHandling(List.of(bogus)));
assertTrue(ex.getMessage().contains("does not exist"));
}
@Test
@DisplayName("validation exception type is not an IOException for bad input")
void validationThrowsIllegalArgumentNotIOException() {
ProcessExecutor exec = qpdfExecutor();
Exception thrown =
assertThrows(
Exception.class,
() -> exec.runCommandWithOutputHandling(List.of("qpdf", "x\ny")));
assertInstanceOf(IllegalArgumentException.class, thrown);
assertFalse(thrown instanceof IOException);
}
}
}
@@ -0,0 +1,10 @@
# Widget Inventory Report
This report lists current stock levels for each warehouse.
| Region | Units | Status |
|---|---|---|
| North | 1200 | OK |
| South | 950 | Low |
| East | 1430 | OK |
| West | 875 | Low |
@@ -0,0 +1,74 @@
%PDF-1.4
%“Œ‹ž ReportLab Generated PDF document (opensource)
1 0 obj
<<
/F1 2 0 R /F2 3 0 R
>>
endobj
2 0 obj
<<
/BaseFont /Helvetica /Encoding /WinAnsiEncoding /Name /F1 /Subtype /Type1 /Type /Font
>>
endobj
3 0 obj
<<
/BaseFont /Helvetica-Bold /Encoding /WinAnsiEncoding /Name /F2 /Subtype /Type1 /Type /Font
>>
endobj
4 0 obj
<<
/Contents 8 0 R /MediaBox [ 0 0 612 792 ] /Parent 7 0 R /Resources <<
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
>> /Rotate 0 /Trans <<
>>
/Type /Page
>>
endobj
5 0 obj
<<
/PageMode /UseNone /Pages 7 0 R /Type /Catalog
>>
endobj
6 0 obj
<<
/Author (\(anonymous\)) /CreationDate (D:20260603003133+01'00') /Creator (\(unspecified\)) /Keywords () /ModDate (D:20260603003133+01'00') /Producer (ReportLab PDF Library - \(opensource\))
/Subject (\(unspecified\)) /Title (\(anonymous\)) /Trapped /False
>>
endobj
7 0 obj
<<
/Count 1 /Kids [ 4 0 R ] /Type /Pages
>>
endobj
8 0 obj
<<
/Filter [ /ASCII85Decode /FlateDecode ] /Length 500
>>
stream
Gas1[9i&Y\%))C:pc(6t3;pQ8%0T@tD+-Nf,MP8j(><jDruNO^VsZ'm=ue7^Ia=;l(`h-+jUU3'Knh#YjH-MEIFj6r!oqf+FjMh;(rCq>R4tMCHL4NI*\1+UNiI'V9NC%VeJKn/YI0J];XQt&X83?=ihrg<*Mcn1n!1nWcDaQPe\P"9gnJuHl(jf]JQgZ[,&^uobI4QF',k"*^S)3c;)GMWC(T'=",ErnS#U=YCUN0&q4+*KmK1Zd*NI\GQDiZUG7;PTja8lulb"\PWWO#WcfI[ZB:6s*3g$be%?JH<b[c(?4J1!2SLDmbmkh8.c&CU)B?Xu,cp$<hP@=fLc>(n`oaEJ[XE'%QW=HE04M<,;ERm[MS=uYF=nN3jG'f@#?O48Ia,6Y-3m&tTWVq1?DeiBkp.Ug*;lVZX`Z=P.eklHhNV;!R_?QOuoeJ<0%7idG7GM8boU$^>N.N,2^;25]0Z8M<<]XMCct>noC'Qfb?`*[Mo+,F9#>t~>endstream
endobj
xref
0 9
0000000000 65535 f
0000000061 00000 n
0000000102 00000 n
0000000209 00000 n
0000000321 00000 n
0000000514 00000 n
0000000582 00000 n
0000000862 00000 n
0000000921 00000 n
trailer
<<
/ID
[<ee376a59c53e3f2af87024f65fd4222c><ee376a59c53e3f2af87024f65fd4222c>]
% ReportLab generated PDF document -- digest (opensource)
/Info 6 0 R
/Root 5 0 R
/Size 9
>>
startxref
1511
%%EOF
@@ -0,0 +1,222 @@
Intro paragraph for section 1.
| Name | Qty |
|---|---|
| alpha | 101 |
| delta | 201 |
# Section 2 Heading
| Name | Qty | Price |
|---|---|---|
| alpha | 101 | charlie |
| delta | 201 | foxtrot |
| golf | 301 | india |
## Section 3 Heading
| Name | Qty | Price | Region |
|---|---|---|---|
| alpha | 101 | charlie | 3 |
| delta | 201 | foxtrot | 13 |
| golf | 301 | india | 23 |
| juliet | 401 | lima | 33 |
Intro paragraph for section 4.
| Name | Qty | Price | Region | Status |
|---|---|---|---|---|
| alpha | 101 | charlie | 3 | echo |
| delta | 201 | foxtrot | 13 | hotel |
| golf | 301 | india | 23 | kilo |
| juliet | 401 | lima | 33 | november |
| mike | 501 | oscar | 43 | alpha |
# Section 5 Heading
| Name | Qty |
|---|---|
| alpha | 101 |
| delta | 201 |
| golf | 301 |
| juliet | 401 |
| mike | 501 |
| papa | 601 |
| Name | Qty | Price |
|---|---|---|
| alpha | 101 | charlie |
| delta | 201 | foxtrot |
# Section 7 Heading
Intro paragraph for section 7.
| Name | Qty | Price | Region |
|---|---|---|---|
| alpha | 101 | charlie | 3 |
| delta | 201 | foxtrot | 13 |
| golf | 301 | india | 23 |
## Section 8 Heading
| Name | Qty | Price | Region | Status |
|---|---|---|---|---|
| alpha | 101 | charlie | 3 | echo |
| delta | 201 | foxtrot | 13 | hotel |
| golf | 301 | india | 23 | kilo |
| juliet | 401 | lima | 33 | november |
| Name | Qty |
|---|---|
| alpha | 101 |
| delta | 201 |
| golf | 301 |
| juliet | 401 |
| mike | 501 |
# Section 10 Heading
Intro paragraph for section 10.
| Name | Qty | Price |
|---|---|---|
| alpha | 101 | charlie |
| delta | 201 | foxtrot |
| golf | 301 | india |
| juliet | 401 | lima |
| mike | 501 | oscar |
| papa | 601 | bravo |
| Name | Qty | Price | Region |
|---|---|---|---|
| alpha | 101 | charlie | 3 |
| delta | 201 | foxtrot | 13 |
# Section 12 Heading
| Name | Qty | Price | Region | Status |
|---|---|---|---|---|
| alpha | 101 | charlie | 3 | echo |
| delta | 201 | foxtrot | 13 | hotel |
| golf | 301 | india | 23 | kilo |
## Section 13 Heading
Intro paragraph for section 13.
| Name | Qty |
|---|---|
| alpha | 101 |
| delta | 201 |
| golf | 301 |
| juliet | 401 |
| Name | Qty | Price |
|---|---|---|
| alpha | 101 | charlie |
| delta | 201 | foxtrot |
| golf | 301 | india |
| juliet | 401 | lima |
| mike | 501 | oscar |
# Section 15 Heading
| Name | Qty | Price | Region |
|---|---|---|---|
| alpha | 101 | charlie | 3 |
| delta | 201 | foxtrot | 13 |
| golf | 301 | india | 23 |
| juliet | 401 | lima | 33 |
| mike | 501 | oscar | 43 |
| papa | 601 | bravo | 53 |
Intro paragraph for section 16.
| Name | Qty | Price | Region | Status |
|---|---|---|---|---|
| alpha | 101 | charlie | 3 | echo |
| delta | 201 | foxtrot | 13 | hotel |
# Section 17 Heading
| Name | Qty |
|---|---|
| alpha | 101 |
| delta | 201 |
| golf | 301 |
## Section 18 Heading
| Name | Qty | Price |
|---|---|---|
| alpha | 101 | charlie |
| delta | 201 | foxtrot |
| golf | 301 | india |
| juliet | 401 | lima |
Intro paragraph for section 19.
| Name | Qty | Price | Region |
|---|---|---|---|
| alpha | 101 | charlie | 3 |
| delta | 201 | foxtrot | 13 |
| golf | 301 | india | 23 |
| juliet | 401 | lima | 33 |
| mike | 501 | oscar | 43 |
# Section 20 Heading
| Name | Qty | Price | Region | Status |
|---|---|---|---|---|
| alpha | 101 | charlie | 3 | echo |
| delta | 201 | foxtrot | 13 | hotel |
| golf | 301 | india | 23 | kilo |
| juliet | 401 | lima | 33 | november |
| mike | 501 | oscar | 43 | alpha |
| papa | 601 | bravo | 53 | delta |
| Name | Qty |
|---|---|
| alpha | 101 |
| delta | 201 |
# Section 22 Heading
Intro paragraph for section 22.
| Name | Qty | Price |
|---|---|---|
| alpha | 101 | charlie |
| delta | 201 | foxtrot |
| golf | 301 | india |
## Section 23 Heading
| Name | Qty | Price | Region |
|---|---|---|---|
| alpha | 101 | charlie | 3 |
| delta | 201 | foxtrot | 13 |
| golf | 301 | india | 23 |
| juliet | 401 | lima | 33 |
| Name | Qty | Price | Region | Status |
|---|---|---|---|---|
| alpha | 101 | charlie | 3 | echo |
| delta | 201 | foxtrot | 13 | hotel |
| golf | 301 | india | 23 | kilo |
| juliet | 401 | lima | 33 | november |
| mike | 501 | oscar | 43 | alpha |
# Section 25 Heading
Intro paragraph for section 25.
| Name | Qty |
|---|---|
| alpha | 101 |
| delta | 201 |
| golf | 301 |
| juliet | 401 |
| mike | 501 |
| papa | 601 |
@@ -0,0 +1,169 @@
%PDF-1.4
%“Œ‹ž ReportLab Generated PDF document (opensource)
1 0 obj
<<
/F1 2 0 R /F2 3 0 R
>>
endobj
2 0 obj
<<
/BaseFont /Helvetica /Encoding /WinAnsiEncoding /Name /F1 /Subtype /Type1 /Type /Font
>>
endobj
3 0 obj
<<
/BaseFont /Helvetica-Bold /Encoding /WinAnsiEncoding /Name /F2 /Subtype /Type1 /Type /Font
>>
endobj
4 0 obj
<<
/Contents 13 0 R /MediaBox [ 0 0 612 792 ] /Parent 12 0 R /Resources <<
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
>> /Rotate 0 /Trans <<
>>
/Type /Page
>>
endobj
5 0 obj
<<
/Contents 14 0 R /MediaBox [ 0 0 612 792 ] /Parent 12 0 R /Resources <<
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
>> /Rotate 0 /Trans <<
>>
/Type /Page
>>
endobj
6 0 obj
<<
/Contents 15 0 R /MediaBox [ 0 0 612 792 ] /Parent 12 0 R /Resources <<
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
>> /Rotate 0 /Trans <<
>>
/Type /Page
>>
endobj
7 0 obj
<<
/Contents 16 0 R /MediaBox [ 0 0 612 792 ] /Parent 12 0 R /Resources <<
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
>> /Rotate 0 /Trans <<
>>
/Type /Page
>>
endobj
8 0 obj
<<
/Contents 17 0 R /MediaBox [ 0 0 612 792 ] /Parent 12 0 R /Resources <<
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
>> /Rotate 0 /Trans <<
>>
/Type /Page
>>
endobj
9 0 obj
<<
/Contents 18 0 R /MediaBox [ 0 0 612 792 ] /Parent 12 0 R /Resources <<
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
>> /Rotate 0 /Trans <<
>>
/Type /Page
>>
endobj
10 0 obj
<<
/PageMode /UseNone /Pages 12 0 R /Type /Catalog
>>
endobj
11 0 obj
<<
/Author (\(anonymous\)) /CreationDate (D:20260603005358+01'00') /Creator (\(unspecified\)) /Keywords () /ModDate (D:20260603005358+01'00') /Producer (ReportLab PDF Library - \(opensource\))
/Subject (\(unspecified\)) /Title (\(anonymous\)) /Trapped /False
>>
endobj
12 0 obj
<<
/Count 6 /Kids [ 4 0 R 5 0 R 6 0 R 7 0 R 8 0 R 9 0 R ] /Type /Pages
>>
endobj
13 0 obj
<<
/Filter [ /ASCII85Decode /FlateDecode ] /Length 1007
>>
stream
GauHKgN)$k&:O:Sn<io8&IUqd[51N1e*/]GIR%m`>DZmeMUK]*(BdhV"2[X,j0;*\%_E*0HXJRc`<Z.PlPcH-)*5ON#Qr4-#h+U(ll5egXa[FW7^<l-+Mko1pU57W'%??d'1Q-,]\qWS80;JOp0HFGa1T[+D>q%4S#/VM`O[Rh4i5T.Phi,$]aT$05!q:PBuCQU/.p2]@1koN*Tb*K9k5G\ht+Dr\K=+8\NZ"alMaOEo**@OK:8-.O1X3-?Gg`@m3%,ti3'">T-&c=M&Wuu?cDbGDp.gO<KniopM)A\M"ajn9N]6p@tefHlX67:iW3SE/[O\LV1TKe.i9es,%kWY+1-s,ft.u27]:^>F0!r6&&CLC$tM?fIR"M37["/*k9@YkpKKSS@Np"1/4#R]`I^(g*1Sc,9L3Qt6N(T@F`A>oBGgL-!r#Y4)R\G`D,iVtFeJUE9u4iuUQ?D%C&SB4kkp.>D5tii>nDKJ"Y07jANhOb$R(_$=U7Stjs)-/KZ(6IBm`6<Z:IO1/g<Q6n^@fdK*]SO#g&Wj4B)g1,#O?30",57V_ONl0p`%uQN]'EAZV$475Z"'DApH2)Sg6UQ75XC?4^'<&6&d?YC`OF9F3eE;WIMt@a?'q)c\3oWQ_BD]d`4=O\,'aSUFV4HsE7)O:lPOi6Ht1,[dTm6JqQM6es[?9G4VC(YieF*)uM6[SZT#[TN2,BIir^1;/Ku"E1laH[\"M+F'8=n0!;?_#hHd!hkpV0u\Ij\4VC-km.[F&JhGmbc`6@I1=mf>u<3;i.Bh4+MFJ"H:.XWUQX6%LU(sg4Tt$_":5`p.2,gZkpUfdg,Hd(qR1)9\ltF2^8b5,,XbY%VfhD52O7A.c.u]dhTc5t%0<0L5E38`Bq+i;"%J7kcc#@i)@okdN-qiaA"33DdPgPrUs7;j%+47_cnGN@&ug9/dqGOAHA4f.N*&guHspf?E;GIc-Wt>:<(m1AmcS_Zc2VlEI'_S_>@#!MF^m1$$<mB1`gt\X~>endstream
endobj
14 0 obj
<<
/Filter [ /ASCII85Decode /FlateDecode ] /Length 987
>>
stream
GatU3gN&c;&:O:SkV;H,eI!J\f?X"LSPIZ'"4Y.F#oAAY?N.Y?8Iu:s)eV;,aNG1Lea>G$!MSc\rU6m7jF'AOI09V1,^TSTi$A+f4sZVQ%2^>uj$40\AI>WbZo9q!NL^CS9P3E9X.:XEMb$a/X"qH9hN6fY,]D!OEOVT,722%RJnVqK4f'[4d3(/hbJWJRs,>>28jk5A:nCl/%FlC#LntsAC5cE$q`QO_@Pk%Lp>7V23")NNrFkcXfuoI=SJJ-`g)_+K\@,SQ@8ORGg(_(B.[u[WXD%Q8@I";9d(]M62K_)Q+-<\kHY5,)o99%BB2bcQVkd:+MZVH=$Z`S@BhH]-XfPANk@qBN.:Jc'?.Kn\p7(aPI<Cb@g:XRZ9Km+RD-e6S\N*kQMA?P5>QOI(du7U]WDrNNPW%d"8P_j^Y8d%G<t1\CJ_5-m+>,'JJLXrV.UD!ah19&9b#*fgR2o730<9)QX)X0"EK6u;mBJs3/fl\d&K$B3bXI5R>F*E,^\%+b?0o8s$o.CuX4uDAT_?G0(p4N3La,8qfi'j?e`e88KrZ8*NDgc:62'icCb],4B]Z"SO&e/BRC*C$2PmbSKAjc\$FLD8?&qHVH3G4p834/77n[%()-p/JDRBapcO2b\,3dW%#U9u!ilSkd%2'rr&g[u".1O]W%0J<@#(ptUD;%0:^.^ls:.#.V6NgR[`2\RUW!k$-*GZIf3DO<K8np=hCA>BiB_Mu"=LVGlO\1Z]0pF(@UaiVQ=dh?h5kfQ_gj>$jHW$8TCt:C.jVU98ne.XoiMtoV;q8XhnlOE;3a]=dGJ8KL&+Khj"&P4Q=P-:nHsDprY"B4+#A3dF,<W=NC>;b&gf(sT\Ts*iH)f!KhfHV;\mMSHej.9.XrBG41_<:b)MG;-Y(:&nCKS0F]r&CGl>EfMc0td\0GHScl3Lr;?OUoo6619Qdr)S8S);]7L5rmWW&(5t4Dil"USDhhY/Y=!=!X:85**$:~>endstream
endobj
15 0 obj
<<
/Filter [ /ASCII85Decode /FlateDecode ] /Length 976
>>
stream
GatU3gN&c;'R]XVkgAc"70atdYFXp#3h7VV#7/=-%N$"G?N.YO&iX#cVsS_l_5i8)eum:1-)(7`mXJ7LnhqY0hGZal,We=q^e"$M]Lu;_=4AoIN$QOjOF.@i*Sg!_rLL=-'?kk;m;M&R`$D>'HCL7<qMHOgP.\NgCm:Zf5tLAgarh6YCKLONS%?!`3T6F7YKNJ(dBFh(U1"e7D6YdC:71/`?cjR"QrKtsGg*:'=Ck>]$A,-S`H1I/1T]4\B3=gQ9:I4j9f]+af*Z7*GH5H+;j73Z-T#DAcgpO<SFuA['l7@MF*6c^<pB]o84rG$&I@Cs3`KIG,N/bG0f<k/M@gUKb"%\W-lZ#AU+i=gUKkm46K-b>W(/'b?XF9+X+'-Dqi"+1U7?6=ZRJNBSJ6ojDTa+]o4RF^26C+i0s(9G2o;$@RNbs\(f75],L(SVpI0M!'7JVhZ"$t/%kb1+)F\p54/@jpVJF2+)7^F=gVcp8(h*m%F#'p9c%9Ep9?b7g^F5>(>I3t^d_"f'aue%^E\X4kN:g\P[;sC=g3h,nD/n9*`94uoM?sBCL<kr$AOqh:,Y&c90,npSDXN*bEfY(k^0LZ_\I@Y+L2nn64-qsuX5D'=o'B2"8rCsRJ?h/-4Ur6khb;ph%X"8-mXTUU5-!Uh$iVuRi1.=>4;XilfMPT4]&FC.U%DI(c6uf3*XPJP2MRdq@Ag3uYF().nqLFAE2Nih'(^p\5FnSNa>n+cI?U65A61_N1<9ZXH4Xr)Fgq(6md3H9[MQ3Q^%aq>E?XM:O:RKGjc+QV2MB!.I.17QpSt]X[C*6ap.`#'o;8F(ebZ.VMM6k`BkZB\I,2MNoX]J"k]Qd";,@($aWX&29q"6k;>Let)@]QHm:i/lf[DB_&U@ROq-0MLQp;@j+]?$"E)brXVun52AVQo$"[a7@!?!LT*>$&:5X_.R;9)'$l-S.>WjLQIVTo.JU%(H\,-*pl_kP9~>endstream
endobj
16 0 obj
<<
/Filter [ /ASCII85Decode /FlateDecode ] /Length 1024
>>
stream
GauHKflEQ9'Rf^WgrHc4<#\/Qm7c-rFIIq+TFSD%Z#L'6o(Nl&^_!k0ds=.%-s&pt#i0QBKE91*<^/MJJCcfoHA_e;aL?\&_BAj]Dt;HW$6(8Ql\%O7-5%uT8<qrc9!'@hZ!'`C3a&iu\ZDZi+5lF*E@>0ZIL#\l)(+[ABOTNKmq!2F,S:.F93Z6QIu[nf\Q[qCE`Y-=.8k$67iCR!7=El+50a@f&b^6'Yg'mqlcRQ-5j:%[K@'5!A_m0L)&VlW1T50^X)"3Ma3,U1P8Di$>uPZ)Zj_igBc0lm]WE0#e>*M53(Yh6\9jHYh6B+3bM\b[Q9j'O_9:!:W0<K\bX;'ZV[%iHrL=:rH.Wh5O-II8-Rh3..Sd^\dNItUUd<nli^`$=A@?\WW4/7Wnp>$Ijm,#(bKn_D?UYMGJ7LLqC0\[$l<8U*O5^GnN;!N6YB'?d[d8DAPt^>+E:r0<2$'VL/TtuNg;C@5iM#%KdS-GU->0]`/20eLXVW#qnjsU.P:Xj&=enVE[mpqDCn2!qDRuQhI/#cZ-#m6`U9'SI4"9\<WSWoVPaf@]<NRl?FrTPOd:V_=>"r:4p.\W>IK_$#C,EkU#uO1g6*HoUK':XiJjtQfI'Sdc)B3J1eqgGJ+l?"$<J+^?VHgiq^90uD"@6ar+/_WhKG+8t!Slo>//&C$^f'TA!2:K!0pd;^Zif)c[:f!C/8hNKc7+'+1WbOFKr9T7c_G$s*O634O2Zs:"iN>j,pWU+:kEQ>5YbF=$27*59iJE<glrr=EAMssR&N"lVT1:*a,OcI=8\"6/>A//7(KsL\pUG.G[+JB-r8t;Sih+(W.A'8Q*LZK*I9WjCLjEt>lT%2RFJZg0Ps:5c,HdK#tSNI:#Rp!]O$Ic=T%K0[._E_Fcjn>OcM,iSAf9iF7,,+Qe:@o'Y4o_:54t*l+TAg<i_F?ke9:\G<n<b*t@"_`_;08ft@[9;qtKC#H0Z6cJ+d)MdidY-OFV1K"=!=K=^?hK-Q.s+?\[I98%IXRiAI.!:AVTf`~>endstream
endobj
17 0 obj
<<
/Filter [ /ASCII85Decode /FlateDecode ] /Length 1075
>>
stream
GauHKgN&c;&:O:SkV;H,6&Q<FCM0)BVUK5trr=4p-3<-<P.#jI+b[BoQ`]$,GaorC,Bfp>Z[5)3-pL]0bpl':D91fR,o"F49.1/bfmFt32ljt6eO[j7!K]\nZ:^QQASu[l@3no(6*HL'@3I!Qfi7%,.N\.>A91O)D\n5*o@JWSOI@ng5p/"_I`g*Y$j0+5PWaso'25ltcglFd,QIaO'dPO0Yr\h5/,=m\PiP:P+Gg0&N;_5iLN$]BD.#VL+h,tSe\fL&,;+2fg&%\pAJiV#eVhq@ro%%/`[(&8n29YiYd>bDF^Bg[AVf/n>[FE(au"ZAe&>%Q[7/n3-QhsPXuPbp#<$d"UR8u-nBnr!Q$Wjd1;9WN/,KG8])Ca5rrDrI'Ue'*I0C%pn+f`Hd"4sB_&2*)S$pX=F/6"DC"TU`bMtd/m7u]?]=J5L[SNBuE:6Ri,NmW7.*qUp)97Tt+a84b\os`Ti8/uRD<mg2;n&q)23L#\f:[QJk+sYYhIC_O\TNS/;3<qfN@GE^L#jl^i&)"7_/%Cb;-M;;ooT&`R;C&\WTZ4"\YIcTZDa;0CKTX+?@ra\94$H:Q<l)r*NHFJWjMr7[tCBb^PX,G]H!fs0g3,5^'AZ%<4Zqp(JCd+;^lkoDJ%q(Bb84V=:PCgOtM#Rr-=oHW&DmO?/!mYrV;g]MR?14cGuH_H*]73!34(()[A.)c<V3:@I>lhrl*Xrn(a0\,%4%*k^rL\k3%6ako`&A3MoN^CXV77D!Qg<S_OEMIh5(T*nFt'&4]_38'Q<s*V7!;A%5-@W.E/(oCu;?^7kb[#R/-P*GF@KHDm6E*dLMg/4Q$W(<Ta[fVpGB5td>%=j9Ge+0B@dZDHELb2fDU(,iiXXRl^*U,U#Fld!R72UIpK<KU`cn&R/_eYS//1PE/$d7p:t-BP62fL<3/QQ4%=NoD?Tlf:A,GpJc<]*uYaL6fSAqm"-E.#%h]idd9$6\DZKKp8Vd[>GWu#-DXi653a?U$fqCs18gt(5/#U%!u,gkMK;>OT_/='S:kn,iD@u]<8-rlTRu'+\7L$U7-#!5-5\WQn(n:q@-p24u!~>endstream
endobj
18 0 obj
<<
/Filter [ /ASCII85Decode /FlateDecode ] /Length 590
>>
stream
GasbXgMWc?&;KY!MRgt!"n`^Bf\9$c*`Q-Sb6tfd8P(q-dT1eng=QV-!OG*\kiWo2AGBdKLqI^%h,XNB)-gDk+GFVBa?g*a%:!J\97Vc8-jH8>8P>0S@Wr)o*"Hu<6qPp`EF:M3'l4@S\c2]`&[J%dO@5p$<(([H6:SlB;FS8L`&pO%6Y\V"/O7F]E%L@d.%>L@"4bK_XXs&";n_.W^Nr847HH,X@$.pC"4bYC?9RH,dN?h8<BTH9l89:QMq:es\ePUpFRc4?hV'h3b4`sE_b4r->a+8/!?Q=D\F41Nc@'B`';4XgMeh!;U23C+>AbMQD-o--lJ/":m#(Yt,X5(`1KL,GTMBLlr6=-1mi#k.-Ou\\T4$Pun2dEU(\?$&;1@T4dm^t!KuOD-U@N_AMr"&uVpsGm,+8I7B*f!%9.o4cC1<Nr^$d5!dH-#UIr,n6Z;JUFYWCTVA\,or>a[CZ12(hd>1*0bU`k2-MXo1[Gor#kmXGIM'#R49X#NSAOpdf'0ilUH4:M(^Snc3;m/+><UlJ1jSN&ZuoQ68F8`c/Y-e0[!aljuioC5fR7-VUAX`%g<0P5J7"CFlQJT1~>endstream
endobj
xref
0 19
0000000000 65535 f
0000000061 00000 n
0000000102 00000 n
0000000209 00000 n
0000000321 00000 n
0000000516 00000 n
0000000711 00000 n
0000000906 00000 n
0000001101 00000 n
0000001296 00000 n
0000001491 00000 n
0000001561 00000 n
0000001842 00000 n
0000001932 00000 n
0000003031 00000 n
0000004109 00000 n
0000005176 00000 n
0000006292 00000 n
0000007459 00000 n
trailer
<<
/ID
[<ba877f458edd6b06b69a7e843c67586d><ba877f458edd6b06b69a7e843c67586d>]
% ReportLab generated PDF document -- digest (opensource)
/Info 11 0 R
/Root 10 0 R
/Size 19
>>
startxref
8140
%%EOF
@@ -0,0 +1,25 @@
# Lorem Ipsum in Two Columns
## 1. Origins
Lorem ipsum dolor sit amet consectetur adipiscing elit. Sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.
## 2. Structure
Ut enim ad minim veniam quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.
## 3. Usage
Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur.
## 4. Variations
Excepteur sint occaecat cupidatat non proident sunt in culpa qui officia deserunt mollit anim id est laborum.
## 5. Typography
Curabitur pretium tincidunt lacus. Nulla gravida orci a odio. Nullam various turpis et commodo pharetra est.
## 6. Conclusion
Nunc nonummy metus. Vestibulum volutpat pretium libero. Cras id dui. Aenean ut eros et nisl sagittis vestibulum.
@@ -0,0 +1,74 @@
%PDF-1.4
%“Œ‹ž ReportLab Generated PDF document (opensource)
1 0 obj
<<
/F1 2 0 R /F2 3 0 R
>>
endobj
2 0 obj
<<
/BaseFont /Helvetica /Encoding /WinAnsiEncoding /Name /F1 /Subtype /Type1 /Type /Font
>>
endobj
3 0 obj
<<
/BaseFont /Helvetica-Bold /Encoding /WinAnsiEncoding /Name /F2 /Subtype /Type1 /Type /Font
>>
endobj
4 0 obj
<<
/Contents 8 0 R /MediaBox [ 0 0 612 792 ] /Parent 7 0 R /Resources <<
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
>> /Rotate 0 /Trans <<
>>
/Type /Page
>>
endobj
5 0 obj
<<
/PageMode /UseNone /Pages 7 0 R /Type /Catalog
>>
endobj
6 0 obj
<<
/Author (\(anonymous\)) /CreationDate (D:20260603021636+01'00') /Creator (\(unspecified\)) /Keywords () /ModDate (D:20260603021636+01'00') /Producer (ReportLab PDF Library - \(opensource\))
/Subject (\(unspecified\)) /Title (\(anonymous\)) /Trapped /False
>>
endobj
7 0 obj
<<
/Count 1 /Kids [ 4 0 R ] /Type /Pages
>>
endobj
8 0 obj
<<
/Filter [ /ASCII85Decode /FlateDecode ] /Length 861
>>
stream
Gat=i>Ak00&;B$5/*8Q/Z)fmNp(^2^OK'MC[dT6#Oq.&b^4c4;1No6>+D%TC.n+fice+l9R0[L%'(o<DJBpB+r$jsRcu23$RjI].E7rS*BB"3t\qRKaS-SYt[_<-]),Cp@'$i;6dQuJ#Q*+lGY?X<phgEWPa?op]_1g@kf_>sQ?dQP@AUa.A:T=#%ZIIl18hB7Zf<piVi-V+;]ct^jS<=Z4l:IOd[,o"%q.hK[[k]r1!^oX'%44os7T9[m%YrbZ3KU/;aR0$cV$;D._".OT8)jeEBKU'l<A`'aD&R'L"Bo]Q)R+=cKOEld8s#M0ZYZN4[lRm1*F6^d9-Gfl5>I(hVU+?`]UK%;aabAH:q>9NUGQq^.=u])Wt-ETI))C'a[FE@elj!RSrf2Q'F>URAO.C,!DneTPqrj#4e2kb9%1"4qfZ)"#&^0j9nHQ?9nF!j7mVPP5\*Uq'_jMVS]9%`kQB\8*AF_bpr/hGj;HCUOSQU-%5:6S79Ud\b!*tPbr_'pCr$Ea#(FYP31NFhSX.-("1M:$cgH#hX8L(2]R3Q>'BYHCS%pI!;=WdJp,'ii[`QPZ_9mcd\baZ2U(_;c\-p,8EoIEpQ*lstL>]LE;C#\dLnT2R:)BM-fTc['3_He[U,k'!Bo".uERd>SkhRj^J+koSIrZ_dEf_5L'/1h.`+DTK(R:P,WH)h5\se=SZ"L/5b8b..,e/E\o+4YQ+*im^C>AERG/TieEK\)#>U@HXnJ,H0A9-MqhhkDp8%.6Lr,OrK*lih;B<-opZ8%EU?,$r^jmCDAQ`-0/-8/`[p]7Fm%0f:E&S*FV)DX2>#q\bRqA=^_`43#8EA$u%8r6F5`rc9>K@q>E4q^*~>endstream
endobj
xref
0 9
0000000000 65535 f
0000000061 00000 n
0000000102 00000 n
0000000209 00000 n
0000000321 00000 n
0000000514 00000 n
0000000582 00000 n
0000000862 00000 n
0000000921 00000 n
trailer
<<
/ID
[<21a9fbd0a0991a91b6e6e2db0856056e><21a9fbd0a0991a91b6e6e2db0856056e>]
% ReportLab generated PDF document -- digest (opensource)
/Info 6 0 R
/Root 5 0 R
/Size 9
>>
startxref
1872
%%EOF
@@ -0,0 +1,62 @@
# Employee Expense Report
Reimbursement Request
EMP-1047
**Report Header**
| Employee Name | Michael Tran |
|---|---|
| Employee ID | EMP-1047 |
| Department | Client Services |
| Report Date | January 20th, 2026 |
| Reporting Period | January 5th16th, 2026 |
| Manager Approver | Laura Simmons |
**Company Information**
| Company | Summit Consulting Partners |
|---|---|
| Company Address | 88 Riverside Plaza, Suite 1400, New York, NY 10069 |
| Accounting Department Email | expenses@example.com |
**Trip Purpose**
The trip was undertaken for client onsite meetings with Atlantic Energy Solutions in Boston, MA.
**Expense Details**
| Description | Amount | Date | Category |
|---|---|---|---|
| Flight (NYC to Boston roundtrip) | $325.40 | January 5th, 2026 | Airline ticket |
| Hotel (3 nights at Harborview Hotel) | $822.75 | January 5th8th, 2026 | Lodging |
| Taxi from airport to hotel | $48.00 | January 5th, 2026 | Ground transportation |
| Client dinner (3 attendees) | $186.20 | January 6th, 2026 | Meals |
| Parking at JFK Airport | $72.00 | January 5th8th, 2026 | Parking |
| Breakfast (per diem not used) | $18.50 | January 7th, 2026 | Meals |
| Description | Amount | Date | Category |
|---|---|---|---|
| Uber to client office | $22.10 | January 7th, 2026 | Ground transportation |
| Printing + presentation materials | $46.90 | January 8th, 2026 | Materials |
| Lunch with client | $39.75 | January 8th, 2026 | Meals |
| Office supplies (notebooks, pens) | $27.60 | January 10th, 2026 | Supplies |
| Mileage reimbursement (client visit in NJ, 42 miles @ $0.67/mile) | $28.14 | January 14th, 2026 | Mileage |
| Team lunch meeting (internal) | $64.30 | January 15th, 2026 | Meals |
Total Expenses $1,701.64
Reimbursement Method
Reimbursement method Direct deposit
Notes
All receipts are attached. Expenses are business-related and comply with company travel policy.
**Approval**
Michael Tran, Employee
Laura Simmons, Manager
@@ -4,7 +4,6 @@ import java.io.IOException;
import java.net.URISyntaxException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Collections;
import java.util.HashMap;
import java.util.Map;
@@ -75,7 +74,7 @@ public class SPDFApplication {
Map<String, String> propertyFiles = new HashMap<>();
// External config files
Path settingsPath = Paths.get(InstallationPathConfig.getSettingsPath());
Path settingsPath = Path.of(InstallationPathConfig.getSettingsPath());
log.info("Settings file: {}", settingsPath.toString());
if (Files.exists(settingsPath)) {
propertyFiles.put(
@@ -84,7 +83,7 @@ public class SPDFApplication {
log.warn("External configuration file '{}' does not exist.", settingsPath.toString());
}
Path customSettingsPath = Paths.get(InstallationPathConfig.getCustomSettingsPath());
Path customSettingsPath = Path.of(InstallationPathConfig.getCustomSettingsPath());
log.info("Custom settings file: {}", customSettingsPath.toString());
if (Files.exists(customSettingsPath)) {
String existingLocation =
@@ -95,7 +95,7 @@ public class ExternalAppDepConfig {
checkDependencyAndDisableGroup(cmd);
return null;
})
.collect(Collectors.toList());
.toList();
invokeAllWithTimeout(tasks, DEFAULT_TIMEOUT.plusSeconds(3));
// Python / OpenCV special handling
@@ -37,8 +37,8 @@ public class LocaleConfiguration implements WebMvcConfigurer {
public LocaleResolver localeResolver() {
SessionLocaleResolver slr = new SessionLocaleResolver();
String appLocaleEnv = applicationProperties.getSystem().getDefaultLocale();
Locale defaultLocale = // Fallback to UK locale if environment variable is not set
Locale.UK;
Locale defaultLocale = // Fallback to US locale if environment variable is not set
Locale.US;
if (appLocaleEnv != null && !appLocaleEnv.isEmpty()) {
Locale tempLocale = Locale.forLanguageTag(appLocaleEnv);
String tempLanguageTag = tempLocale.toLanguageTag();
@@ -51,7 +51,7 @@ public class LocaleConfiguration implements WebMvcConfigurer {
defaultLocale = tempLocale;
} else {
System.err.println(
"Invalid SYSTEM_DEFAULTLOCALE environment variable value. Falling back to default en-GB.");
"Invalid SYSTEM_DEFAULTLOCALE environment variable value. Falling back to default en-US.");
}
}
}
@@ -18,6 +18,7 @@ import io.swagger.v3.oas.models.media.StringSchema;
import io.swagger.v3.oas.models.security.SecurityRequirement;
import io.swagger.v3.oas.models.security.SecurityScheme;
import io.swagger.v3.oas.models.servers.Server;
import io.swagger.v3.oas.models.tags.Tag;
import lombok.RequiredArgsConstructor;
@@ -60,6 +61,15 @@ public class OpenApiConfig {
OpenAPI openAPI = new OpenAPI().info(info).openapi("3.0.3");
// Register a single global "AI" tag so every AI endpoint groups under it in the docs.
// The AI controllers are currently @Hidden, so they don't emit this tag themselves yet;
// defining it here keeps the grouping ready for when those endpoints are unhidden.
openAPI.addTagsItem(
new Tag()
.name("AI")
.description(
"AI-powered document creation, editing, and assistant endpoints."));
// Add server configuration from environment variable
String swaggerServerUrl = System.getenv("SWAGGER_SERVER_URL");
Server server;
@@ -48,7 +48,7 @@ public class AdditionalLanguageJsController {
}
}
// Fallback
return "en_GB";
return "en_US";
}
""");
writer.flush();
@@ -10,6 +10,7 @@ import java.util.Map;
import org.apache.pdfbox.Loader;
import org.apache.pdfbox.multipdf.Overlay;
import org.apache.pdfbox.pdfwriter.compress.CompressParameters;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.springframework.core.io.Resource;
import org.springframework.http.MediaType;
@@ -157,7 +158,10 @@ public class PdfOverlayController {
PDDocument singlePageDocument = new PDDocument()) {
singlePageDocument.addPage(overlayPdf.getPage(pageCountInCurrentOverlay));
File tempFile = Files.createTempFile("overlay-page-", ".pdf").toFile();
singlePageDocument.save(tempFile);
// NO_COMPRESSION: this single-page doc holds a page copied from overlayPdf.
// PDFBox 3.0.7's compressed writer (PDFBOX-6203) drops shared resources imported
// across documents, corrupting overlay fonts. Revert once on 3.0.8.
singlePageDocument.save(tempFile, CompressParameters.NO_COMPRESSION);
overlayGuide.put(basePageIndex, tempFile.getAbsolutePath());
tempFiles.add(tempFile); // Keep track of the temporary file for cleanup
@@ -6,11 +6,9 @@ import java.util.Collections;
import java.util.List;
import java.util.Locale;
import org.apache.pdfbox.cos.COSName;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDDocumentCatalog;
import org.apache.pdfbox.pdmodel.PDPage;
import org.apache.pdfbox.pdmodel.interactive.form.PDAcroForm;
import org.apache.pdfbox.pdmodel.PDPageTree;
import org.springframework.core.io.Resource;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
@@ -262,38 +260,31 @@ public class RearrangePagesPDFController {
}
log.info("newPageOrder = {}", newPageOrder);
log.info("totalPages = {}", totalPages);
// Create a new list to hold the pages in the new order
List<PDPage> newPages = new ArrayList<>();
for (int i = 0; i < newPageOrder.size(); i++) {
newPages.add(document.getPage(newPageOrder.get(i)));
// Snapshot the desired pages before mutating the source document's page tree.
List<PDPage> newPages = new ArrayList<>(newPageOrder.size());
for (Integer idx : newPageOrder) {
newPages.add(document.getPage(idx));
}
// Create a new document based on the original one
try (PDDocument rearrangedDocument =
pdfDocumentFactory.createNewDocumentBasedOnOldDocument(document)) {
// Add the pages in the new order
for (PDPage page : newPages) {
rearrangedDocument.addPage(page);
}
PDDocumentCatalog sourceCatalog = document.getDocumentCatalog();
if (sourceCatalog != null) {
PDAcroForm sourceForm = sourceCatalog.getAcroForm(null);
if (sourceForm != null) {
rearrangedDocument
.getDocumentCatalog()
.getCOSObject()
.setItem(COSName.ACRO_FORM, sourceForm.getCOSObject());
}
}
return WebResponseUtils.pdfDocToWebResponse(
rearrangedDocument,
GeneralUtils.generateFilename(
pdfFile.getOriginalFilename(), "_rearranged.pdf"),
tempFileManager);
// Rearrange in-place on the source document rather than copying pages into a
// freshly-created PDDocument. Copying pages across documents triggers a PDFBox
// 3.0.7 compressed-save regression (PDFBOX-6203, fixed for 3.0.8) where shared
// resource objects (fonts, etc.) imported from the source can be silently
// dropped from the output, producing pages with "font not found" errors.
PDPageTree pages = document.getPages();
for (int i = totalPages - 1; i >= 0; i--) {
pages.remove(i);
}
for (PDPage page : newPages) {
pages.add(page);
}
return WebResponseUtils.pdfDocToWebResponse(
document,
GeneralUtils.generateFilename(
pdfFile.getOriginalFilename(), "_rearranged.pdf"),
tempFileManager);
}
} catch (IOException e) {
ExceptionUtils.logException("document rearrangement", e);
@@ -5,7 +5,6 @@ import java.io.InputStream;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.*;
import java.util.stream.Stream;
@@ -115,7 +114,7 @@ public class UIDataController {
if (new java.io.File(runtimePathConfig.getPipelineDefaultWebUiConfigs()).exists()) {
try (Stream<Path> paths =
Files.walk(Paths.get(runtimePathConfig.getPipelineDefaultWebUiConfigs()))) {
Files.walk(Path.of(runtimePathConfig.getPipelineDefaultWebUiConfigs()))) {
List<Path> jsonFiles =
paths.filter(Files::isRegularFile)
.filter(p -> p.toString().endsWith(".json"))

Some files were not shown because too many files have changed in this diff Show More