Compare commits

...
Author SHA1 Message Date
Reece 8a7667e18d fix 2026-05-08 12:21:35 +01:00
Reece c93891dfe0 ● Harden form-fill analyser with output validators, per-call timeouts, fallback telemetry; raise chunker threshold to 1500 2026-05-08 11:10:27 +01:00
Reece 273865217e improve many to many handling and ui 2026-05-05 22:13:08 +01:00
Reece a969c5035b Add server-side persistence for AI Form Fill entities and workflow templates 2026-04-27 14:21:24 +01:00
Reece 38f5227ba1 Orchestrator stuff 2026-04-23 23:15:05 +01:00
ReeceandClaude Opus 4.7 62e772fd7a Fix review findings on form-fill branch post-merge
Critical:
- Remove orphaned delegate_form_fill ToolOutput from OrchestratorAgent;
  the method didn't exist and OrchestratorAgent(runtime) crashed at boot.
  Also drop unused DocumentExtractorAgent import and KnowledgeUpdateResponse
  from the OrchestratorResponse union. Form fill stays on its own endpoints
  (POST /api/v1/form/ai/*) and is not wired as an orchestrator delegate.

Medium:
- Thread conversation_history through the three form-fill agents.
  Contracts: FormAnalysisRequest, FormFillBatchRequest, and
  DocumentExtractionRequest now carry conversation_history. Agents call
  format_conversation_history() when building prompts, matching the
  pattern from PdfQuestionAgent / PdfEditAgent.
- Add input bounds to form-fill contracts (max_length on strings,
  min_length/max_length on lists and dicts). Caps: 50 files, 500 fields
  per file, 20 documents per request, 500 knowledge entries, 50k chars
  per document text, 8k chars per page text. Stops unbounded prompt growth.
- Replace Literal["fill_result"] / Literal["knowledge_update"] etc. with
  WorkflowOutcome enum values. Adds KNOWLEDGE_UPDATE, MULTI_PROFILE_EXTRACTION,
  and BATCH_FILL_RESULT to both WorkflowOutcome (Python) and
  AiWorkflowOutcome (Java) to keep the "must stay in sync" contract honest.

Low:
- Drop duplicate build_test_settings() helper in test_form_fill_agent.py;
  use conftest.build_app_settings() like the rest of the suite.
- Add test coverage for extract_multiple -> MultiProfileExtractionResponse
  (the two-person detection path).
- ARCHITECTURE.md: clarify where state actually lives
  (localStorage keys on the frontend, no user-scoping) and note that form
  fill is not an orchestrator delegate.

128 engine tests pass. Lifespan boots cleanly with all four agents.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 18:28:56 +01:00
ReeceandClaude Opus 4.7 1c23607fd4 Fix: restore aiFormFill tool-id and engine-api proxy after merge
The -X theirs merge strategy dropped two entries that main doesn't
have but we need:
- frontend/src/core/types/toolId.ts: add "aiFormFill" back to
  CORE_REGULAR_TOOL_IDS. Without this, useTranslatedToolRegistry's
  aiFormFill: {...} entry fails its type check.
- frontend/vite.config.ts: add back the /engine-api → aiEngineTarget
  proxy. Main defines aiEngineTarget but never uses it; our form-fill
  frontend calls /engine-api/api/v1/form/ai/* which requires the proxy
  to reach the Python engine in dev.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 14:29:15 +01:00
ReeceandClaude Opus 4.7 b32a3cf271 Merge origin/main into AI-Form-Fill
Resolution favours main for all conflicts (new orchestrator shape with
artifacts/file_names/conversation_history/resume_with, WorkflowOutcome
enum, RAG, ledger agent, deleted engine/config/.env.example).

Re-wires our three form-fill agents (FormAnalyserAgent, FormFillerAgent,
DocumentExtractorAgent) into the new app layout:
- contracts/__init__.py: adds form-fill exports to __all__
- api/dependencies.py: adds get_form_analyser_agent / get_form_filler_agent / get_document_extractor_agent
- api/app.py: instantiates the three agents in lifespan and registers form_fill_router
- api/routes/__init__.py: exports form_fill_router
- tests/test_stirling_api.py: imports and registers form-fill stubs
- tests/test_stirling_contracts.py: adds back KnowledgeUpdateResponse discriminator test

Form fill remains accessible via its own endpoints (/api/v1/form/ai/analyse,
/fill-batch, /extract). Not wired as an orchestrator delegate — following
main's pattern where orchestrator only routes pdf_edit/pdf_question/user_spec
/math_auditor.

Follow-ups still needed:
- Thread conversation_history into form-fill agent prompts
- Align form-fill response outcomes with WorkflowOutcome enum
- Decide whether engine should return ToolOperationStep plans (main's
  new pattern per #6116) or keep returning fill values directly

All 127 engine tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 13:13:12 +01:00
Reece 80cf2f9a73 Tweaks and updates 2026-04-21 16:59:31 +01:00
James Brunton fdd8bc23ca Change engine/.env to be committed and have .env.local override (#6150)
# Description of Changes
We keep adding stuff to `engine/config/.env.example` and have to
manually update `.env` because of it, which is really clunky, especially
when working on multiple worktrees at once. This PR changes it so that
we just have a committed `.env` file and have an `.env.local` override
to put the actual private keys into, which should make it a bit easier
to manage.

> [!warning]
>
> After this goes in, be very careful for a little while not to
accidentally commit any keys that you've got inside your `.env` file!
2026-04-21 16:18:25 +01:00
James Brunton 6307c7bb28 Allow chat history to be sent to AI engine (#6128)
# Description of Changes
Add an extra parameter to every agent to receive the conversation
history in addition to the current message. This will make it possible
to answer followup questions from the AI without needing to give full
context in your message.
2026-04-21 15:03:10 +00:00
Anthony Stirling 83deaed780 setup RAG (#6146) 2026-04-21 12:42:33 +01:00
cf2c189e0d Add pixel comparison mode to Compare tool (#6109)
Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
Co-authored-by: EthanHealy01 <80844253+EthanHealy01@users.noreply.github.com>
2026-04-20 19:31:07 +01:00
EthanHealy01andJames Brunton 1e4c630ac8 allow deploypr:prototypes comment to spin up the prototypes build (#6144)
Co-authored-by: James Brunton <jbrunton96@gmail.com>
2026-04-20 18:58:33 +01:00
2f09abf89e Change AI engine to execute tools in Java instead of on frontend (#6116)
# Description of Changes
Redesign AI engine so that it autogenerates the `tool_models.py` file
from the OpenAPI spec so the Python has access to the Java API
parameters and the full list of Java tools that it can run. CI ensures
that whenever someone modifies a tool endpoint that the AI enigne tool
models get updated as well (the dev gets told to run `task
engine:tool-models`).

There's loads of advantages to having the Java be the one that actually
executes the tools, rather than the frontend as it was previously set up
to theoretically use:
- The AI gets much better descriptions of the params from the API docs
- It'll be usable headless in the future so a Java daemon could run to
execute ops on files in a folder without the need for the UI to run
- The Java already has all the logic it needs to execute the tools 
- We don't need to parse the TypeScript to find the API (which is hard
because the TS wasn't designed to be computer-read to extract the API)

I've also hooked up the prototype frontend to ensure it's working
properly, and have built it in a way that all the tool names can be
translated properly, which was always an issue with previous prototypes
of this.

---------

Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
Co-authored-by: EthanHealy01 <80844253+EthanHealy01@users.noreply.github.com>
2026-04-20 15:57:11 +01:00
James Brunton d09c843fb2 Fix any type usage in desktop/ (#6033)
# Description of Changes
Follow on from #5949, expanding any type usage ban to the `desktop/`
folder

Also gets rid of a bunch of really verbose desktop logging that I don't
think we really need anymore (or ever needed tbh, most of it doesn't
make sense) because it was using a bunch of `any` typing and wasn't
worth fixing.
2026-04-20 12:42:38 +00:00
Anthony Stirling e51aa16a32 Fix form-fill hang when flattening with empty values (#6143) 2026-04-20 13:12:25 +01:00
Anthony Stirling 4ac10c2f81 Swap thumbnail rendering from PDF.js to PDFium (#6135) 2026-04-20 12:53:56 +01:00
Anthony Stirling 837e04f56f Fix compare tool file selection and other files improvements (#6133) 2026-04-20 12:53:37 +01:00
Anthony Stirling 918e177f6d fix tests caused by streaming changes (#6137) 2026-04-19 18:35:51 +01:00
Anthony Stirlingandaikido-pr-checks[bot] b866dce5c1 AUR publishing workflow (#6132)
Co-authored-by: aikido-pr-checks[bot] <169896070+aikido-pr-checks[bot]@users.noreply.github.com>
2026-04-17 23:12:46 +01:00
Anthony Stirling 31e8420bf2 enable AppImage and rpm distrobutions (#6127) 2026-04-17 22:19:16 +01:00
Anthony Stirling 7b64dfa935 package manager GHA init to allow workflow dispatch testing (#6129) 2026-04-17 15:56:04 +01:00
EthanHealy01 5671ea92bb Chore/remove usage of mantine color scheme (#6108)
Remove instances of `colorScheme === "dark" ?` in the app and rely on
the theme.css' light and dark variables instead.
2026-04-17 14:29:37 +01:00
Anthony Stirling 568ae50db0 Tauri sign fixes for security alerts (#6122) 2026-04-17 11:05:29 +01:00
James Brunton 4b0b6309b0 Prettier 2: Electric Boogaloo (#6113)
# Description of Changes
When I added Prettier formatting in #6052, my aim was to use just the
default settings in Prettier. Turns out, Prettier looks _really hard_
for any config files if it's not explicitly given one, which means that
if a developer has some sort of Prettier config file lying around on
their system, Prettier might find it and use it. Also, Prettier changes
its defaults based on stuff in `.editorconfig` without any good way of
disabling that behaviour explicitly in its config file.

To solve both of these issues, I've introduced a `.prettierrc` file
which sets Prettier's defaults explicitly, and then reformatted all our
code _again_ in Prettier's actual default settings. This should achieve
the aim of #6052 and remove the possibility for it breaking on different
dev computers.
2026-04-17 09:50:16 +00:00
5395266f7d Feat/math validation agent (#6012)
Co-authored-by: James Brunton <jbrunton96@gmail.com>
Co-authored-by: EthanHealy01 <80844253+EthanHealy01@users.noreply.github.com>
2026-04-17 10:36:45 +01:00
James Brunton d4d4ac83e0 Add streaming to Engine orchestrator (#6094)
# Description of Changes
Adds a streaming endpoint to the Java AI orchestrator
(`/api/v1/ai/orchestrate/stream` in addition to the existing
`/api/v1/ai/orchestrate`). This allows the caller to get updates of what
stage of orchestration is being run at the time so UIs can give the user
feedback.

Also contains some dubious Gradle changes to suppress errors coming from
Spotless, when it crashes in Google stuff. I'm not sure if that's
appropriate to add, feel free to ask for changes in review.
2026-04-17 10:01:08 +01:00
Orel Yosupov eb075ac4fa Fix terms and privacy URLs links in Footer component (#6124)
Fix the issue #6104
2026-04-16 15:55:53 +01:00
Anthony Stirling cc2887be4e thumbnail preview fixes windows (#6074) 2026-04-15 23:25:38 +01:00
Anthony Stirling b8d7741273 Cleanup work + stream endpoints to reduce memory usage (#6106) 2026-04-15 15:34:17 +01:00
ConnorYohandJames Brunton 438669d26a Add Taskfile for unified dev workflow across all components (#6080)
## Add Taskfile for unified dev workflow

### Summary
- Introduces [Taskfile](https://taskfile.dev/) as the single CLI entry
point for all development workflows across backend, frontend, engine,
Docker, and desktop
- ~80 tasks organized into 6 namespaces: `backend:`, `frontend:`,
`engine:`, `docker:`, `desktop:`, plus root-level composites
- All CI workflows migrated to use Task
- Deletes `engine/Makefile` and `scripts/build-tauri-jlink.{sh,bat}` —
replaced by Task equivalents
- Removes redundant npm scripts (`dev`, `build`, `prep`, `lint`, `test`,
`typecheck:all`) from `package.json`
- Smart dependency caching: `sources`/`status`/`generates`
fingerprinting, CI-aware `npm ci` vs `npm install`, `run: once` for
parallel dep deduplication

### What this does NOT do
- Does not replace Gradle, npm, or Docker — Taskfile is a thin
orchestration wrapper
- Does not change application code or behavior

### Install
```
npm install -g @go-task/cli    # or: brew install go-task, winget install Task.Task
```

### Quick start
```
task --list       # discover all tasks
task install      # install all deps
task dev          # start backend + frontend
task dev:all      # also start AI engine
task test         # run all tests
task check        # quick quality gate (local dev)
task check:all    # full CI quality gate
```

### Test plan
- [ ] Install `task` CLI and run `task --list` — verify all tasks
display
- [ ] Run `task install` — verify frontend + engine deps install
- [ ] Run `task dev` — verify backend + frontend start, Ctrl+C exits
cleanly
- [ ] Run `task frontend:check` — verify typecheck + lint + test pass
- [ ] Run `task desktop:dev` — verify jlink builds are cached on second
run
- [ ] Verify CI passes on all workflows

---------

Co-authored-by: James Brunton <jbrunton96@gmail.com>
2026-04-15 14:16:57 +00:00
James Brunton 79b5fc72d9 Fix Java formatting (#6114)
# Description of Changes
#6069 introduced formatting issues in the Java, this PR fixes them.
2026-04-15 15:12:04 +01:00
Anthony Stirling 9b158e9944 desktop mobile QR fixes (#6069) 2026-04-15 13:21:45 +01:00
James BruntonandConnorYoh 8d394e60e9 Add tracking system to support optional PostHog tracking in AI engine (#6040)
Co-authored-by: ConnorYoh <40631091+ConnorYoh@users.noreply.github.com>
2026-04-14 18:45:47 +01:00
plind 6d1ef33f02 Fix encrypted PDF unlock modal missing on IndexedDB restore and large files (#6099) 2026-04-14 00:38:42 +01:00
Reece Browne a6704227e0 Fix encrypted pdf handling (#6088)
Fix and improve encrypted pdf handling
2026-04-13 13:20:43 +01:00
Reece BrowneandClaude Opus 4.6 cedf673340 Remove duplicate isPanning state (#6086)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 11:49:09 +01:00
unlair a56c353e2d Fix healthcheck in Docker files when SYSTEM_ROOTURIPATH is specified (#5954) 2026-04-12 22:44:04 +01:00
James Brunton df08ad0749 Add frontend autoformatting and set CI to require formatted code for all languages (#6052)
# Description of Changes
Changes the strategy for autoformatting to reject PRs if they are not
formatted correctly instead of allowing them to merge and then spawning
a new PR to fix the formatting. The old strategy just caused more work
for us because we'd have to manually approve the followup PR and get it
merged, which required 2 reviewers so in practice it rarely got done and
just meant everyone's PRs ended up containing reformatting for unrelated
files, which makes code review unnecessarily difficult. If the PR's code
is not formatted correctly after this PR, a comment will be added
automatically to tell the author how to run the formatter script to fix
their code so it can go in.

This also enables autoformatting for the frontend code, using Prettier.
I've enabled it for pretty much everything in the frontend folder, other
than 3rd party files and files it doesn't make sense for. I also
excluded Markdown because it sounds likely to be more annoying to have
to autoformat the Markdown in the frontend folder but nowhere else. Open
to changing this though if people disagree.

> [!note]
> 
> Advice to reviewers: The first commit contains all of the actual logic
I've introduced (CI changes, Prettier config, etc.)
> The second commit is just the reformatting of the entire frontend
folder.
> The first commit needs proper review, the second one just give it a
spot-check that it's doing what you'd expect.
2026-04-10 17:41:19 +01:00
aikido-autofix[bot] dca5787323 [Aikido] Fix 16 security issues in fastmcp, aiohttp, cryptography and 1 more (#6091)
Upgrade fastmcp, aiohttp, cryptography, and anthropic to fix critical
SSRF/path traversal, header injection, OAuth confused deputy, and DoS
vulnerabilities.

<details>
<summary> 16 CVEs resolved by this upgrade, including 2 critical 🚨
CVEs</summary>

<br>


This PR will resolve the following CVEs:
| Issue |
Severity&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |
Description |
| --- | --- | --- |
|
<pre>[CVE-2026-32871](https://app.aikido.dev/issues/25944204/detail?groupId=70007#CVE-2026-32871)</pre>
| <pre>🚨 CRITICAL</pre> | [fastmcp] Path traversal vulnerability in URL
construction allows attackers to bypass API prefix restrictions and
access arbitrary backend endpoints using unencoded path parameters,
enabling authenticated SSRF attacks. |
|
<pre>[CVE-2026-27124](https://app.aikido.dev/issues/25944204/detail?groupId=70007#CVE-2026-27124)</pre>
| <pre>HIGH</pre> | [fastmcp] OAuthProxy fails to validate user consent
when receiving authorization codes from GitHub, allowing attackers to
exploit GitHub's consent-skipping behavior to gain unauthorized access
to FastMCP servers through a Confused Deputy attack. |
|
<pre>[CVE-2025-64340](https://app.aikido.dev/issues/25944204/detail?groupId=70007#CVE-2025-64340)</pre>
| <pre>MEDIUM</pre> | [fastmcp] Server names with shell metacharacters
can cause command injection on Windows when passed to install commands,
allowing arbitrary code execution through cmd.exe interpretation of .cmd
wrapper files. |
|
<pre>[CVE-2026-34520](https://app.aikido.dev/issues/25944198/detail?groupId=70007#CVE-2026-34520)</pre>
| <pre>🚨 CRITICAL</pre> | [aiohttp] is an asynchronous HTTP
client/server framework for asyncio and Python. Prior to version 3.13.4,
the C parser (the default for most installs) accepted null bytes and
control characters in response headers. This issue has been patched in
version 3.13.4. |
|
<pre>[CVE-2026-34516](https://app.aikido.dev/issues/25944198/detail?groupId=70007#CVE-2026-34516)</pre>
| <pre>HIGH</pre> | [aiohttp] A response with an excessive number of
multipart headers can consume more memory than intended, leading to a
denial of service (DoS) vulnerability through resource exhaustion. |
|
<pre>[CVE-2026-22815](https://app.aikido.dev/issues/25944198/detail?groupId=70007#CVE-2026-22815)</pre>
| <pre>MEDIUM</pre> | [aiohttp] is an asynchronous HTTP client/server
framework for asyncio and Python. Prior to version 3.13.4, insufficient
restrictions in header/trailer handling could cause uncapped memory
usage. This issue has been patched in version 3.13.4. |
|
<pre>[CVE-2026-34515](https://app.aikido.dev/issues/25944198/detail?groupId=70007#CVE-2026-34515)</pre>
| <pre>MEDIUM</pre> | [aiohttp] is an asynchronous HTTP client/server
framework for asyncio and Python. Prior to version 3.13.4, on Windows
the static resource handler may expose information about a NTLMv2 remote
path. This issue has been patched in version 3.13.4. |
|
<pre>[CVE-2026-34525](https://app.aikido.dev/issues/25944198/detail?groupId=70007#CVE-2026-34525)</pre>
| <pre>MEDIUM</pre> | [aiohttp] is an asynchronous HTTP client/server
framework for asyncio and Python. Prior to version 3.13.4, multiple Host
headers were allowed in aiohttp. This issue has been patched in version
3.13.4. |
|
<pre>[CVE-2026-34513](https://app.aikido.dev/issues/25944198/detail?groupId=70007#CVE-2026-34513)</pre>
| <pre>LOW</pre> | [aiohttp] is an asynchronous HTTP client/server
framework for asyncio and Python. Prior to version 3.13.4, an unbounded
DNS cache could result in excessive memory usage possibly resulting in a
DoS situation. This issue has been patched in version 3.13.4. |
|
<pre>[CVE-2026-34514](https://app.aikido.dev/issues/25944198/detail?groupId=70007#CVE-2026-34514)</pre>
| <pre>LOW</pre> | [aiohttp] is an asynchronous HTTP client/server
framework for asyncio and Python. Prior to version 3.13.4, an attacker
who controls the content_type parameter in aiohttp could use this to
inject extra headers or similar exploits. This issue has been patched in
version 3.13.4. |
|
<pre>[CVE-2026-34517](https://app.aikido.dev/issues/25944198/detail?groupId=70007#CVE-2026-34517)</pre>
| <pre>LOW</pre> | [aiohttp] is an asynchronous HTTP client/server
framework for asyncio and Python. Prior to version 3.13.4, for some
multipart form fields, aiohttp read the entire field into memory before
checking client_max_size. This issue has been patched in version 3.13.4.
|
|
<pre>[CVE-2026-34518](https://app.aikido.dev/issues/25944198/detail?groupId=70007#CVE-2026-34518)</pre>
| <pre>LOW</pre> | [aiohttp] When following redirects to a different
origin, the framework fails to drop the Cookie and Proxy-Authorization
headers alongside the Authorization header, potentially leaking
sensitive authentication credentials to untrusted domains. |
|
<pre>[CVE-2026-34519](https://app.aikido.dev/issues/25944198/detail?groupId=70007#CVE-2026-34519)</pre>
| <pre>LOW</pre> | [aiohttp] is an asynchronous HTTP client/server
framework for asyncio and Python. Prior to version 3.13.4, an attacker
who controls the reason parameter when creating a Response may be able
to inject extra headers or similar exploits. This issue has been patched
in version 3.13.4. |
|
<pre>[CVE-2026-39892](https://app.aikido.dev/issues/25637201/detail?groupId=70007#CVE-2026-39892)</pre>
| <pre>MEDIUM</pre> | [cryptography] Non-contiguous buffers passed to
cryptographic APIs can cause buffer overflows, potentially leading to
memory corruption and arbitrary code execution. |
|
<pre>[CVE-2026-34452](https://app.aikido.dev/issues/25944200/detail?groupId=70007#CVE-2026-34452)</pre>
| <pre>MEDIUM</pre> | [anthropic] A time-of-check-time-of-use (TOCTOU)
vulnerability in the async filesystem memory tool allows local attackers
to escape the sandbox directory via symlink manipulation, enabling
arbitrary file read/write operations outside the intended memory
directory. |
|
<pre>[CVE-2026-34450](https://app.aikido.dev/issues/25944200/detail?groupId=70007#CVE-2026-34450)</pre>
| <pre>MEDIUM</pre> | [anthropic] The local filesystem memory tool
created world-readable and potentially world-writable files, allowing
local attackers to read persisted agent state or modify memory files to
influence model behavior. |


</details>

Co-authored-by: aikido-autofix[bot] <119856028+aikido-autofix[bot]@users.noreply.github.com>
2026-04-10 08:54:53 +00:00
aikido-autofix[bot] 5815d0b824 [Aikido] Fix critical issue in axios via minor version upgrade from 1.13.6 to 1.15.0 in frontend (#6092)
Upgrade axios to fix critical proxy bypass and SSRF vulnerabilities in
hostname normalization that could allow attackers to reach protected
internal services.

 There are no breaking changes

<details>
<summary> 1 CVE resolved by this upgrade, including 1 critical 🚨
CVE</summary>

<br>


This PR will resolve the following CVEs:
| Issue |
Severity&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |
Description |
| --- | --- | --- |
|
<pre>[CVE-2025-62718](https://app.aikido.dev/issues/26490690/detail?groupId=70007#CVE-2025-62718)</pre>
| <pre>🚨 CRITICAL</pre> | [axios] Axios fails to properly normalize
hostnames when checking NO_PROXY rules, allowing requests to loopback
addresses (localhost., [::1]) to bypass proxy protections and reach
internal services. This enables proxy bypass and SSRF attacks against
protected loopback or internal endpoints. |


</details>

Co-authored-by: aikido-autofix[bot] <119856028+aikido-autofix[bot]@users.noreply.github.com>
2026-04-10 09:50:05 +01:00
EthanHealy01 dd15412869 use clean 3 card design for landing page (#6084)
<img width="2056" height="1080" alt="Screenshot 2026-04-08 at 1 26
58 PM"
src="https://github.com/user-attachments/assets/e834988b-c3ab-4633-bf15-9fe0457d0029"
/>

<img width="2056" height="1080" alt="Screenshot 2026-04-08 at 1 27
12 PM"
src="https://github.com/user-attachments/assets/adfebd95-ca59-4de0-9336-b1e2dc1dc5fe"
/>
2026-04-09 12:38:46 +00:00
Vibe Stack d5b7af6567 feat(settings): add default startup view and reader zoom preferences (#6073)
## Description of Changes

Adds two new user preferences to the General settings panel, addressing
#5908.

**Default view on launch** - a segmented control (Tools / Reader /
Automate) that controls which left-column tab is active when the app
starts. Previously the app always opened on the Tools tab with no way to
change this. Users who spend most of their time reading PDFs had to
manually switch to the Reader tab on every launch.

**Default reader zoom** - a dropdown (Auto / Fit width / Fit page /
50%–200%) that sets the initial zoom level whenever a PDF is opened in
the reader. Previously the app always applied an automatic
fit-to-viewport calculation.

Both settings are non-breaking. The defaults (`Tools` and `Auto`)
reproduce the existing behaviour exactly, so existing users see no
difference until they change a preference.

### What changed
- `preferencesService.ts` - added `StartupView` and `ViewerZoomSetting`
types plus the two new fields to `UserPreferences` with safe defaults
- `ToolWorkflowContext.tsx` - one-time startup effect that navigates to
the preferred tab on first render (mirrors the existing
`defaultToolPanelMode` sync pattern)
- `ZoomAPIBridge.tsx` - respects the zoom preference before falling back
to auto-zoom logic when a document loads
- `GeneralSection.tsx` - two new controls added below "Default tool
picker mode"; the Select uses `comboboxProps={{ withinPortal: true }}`
so the dropdown renders above the settings modal
- `en-GB/translation.toml` - new keys for labels, descriptions, and
option values

Closes #5908 

---

## Checklist

### General

- [x] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [x] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md)
(if applicable)
- [x] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [x] I have performed a self-review of my own code
- [x] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [x] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [x] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)
<img width="1023" height="747" alt="Screenshot 2026-04-05 185718"
src="https://github.com/user-attachments/assets/6a8bc35a-d813-4ab8-b303-55bdce747a6a"
/>
<img width="1026" height="755" alt="Screenshot 2026-04-05 185620"
src="https://github.com/user-attachments/assets/d2c45134-ed32-4332-a193-1a96837ba2a3"
/>


### Testing (if applicable)

- [x] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md#6-testing)
for more details.
2026-04-09 10:30:19 +00:00
James Brunton 74e62b86a2 Add prototypes folder to test new functionality in (#6081)
# Description of Changes
Add prototypes folder to test new functionality in. This build of the
app is spawnable with `npm run dev:prototypes`.

Currently just contains a very developer-y chat interface to help us
develop & explore the AI backend before we make the frontend for it for
real.
2026-04-09 08:21:07 +00:00
James Brunton f9575c06fb Add Java orchestrator to connect to the AI engine (#6003)
# Description of Changes
Add Java orchestration layer which can connect and go back and forth
with the AI engine to get results for the user. It's expected that the
AI engine will not be publicly available and this Java layer will always
be in front of it, to manage sessions and auth etc.
2026-04-09 08:04:38 +00:00
b53bd952db Fix/desktop open with tool access (#6056)
## Description

Fixes #6029 - Additional selection in windows client no longer necessary

## Problem
When opening PDF files in the Windows desktop client using "Open with",
the file displays properly but users had to manually select it again in
the workbench before any PDF tools (merge, compress, crop, compare,
etc.) become functional.

## Root Cause
Files opened via "Open with" were added to FileContext but **not
selected** (missing `selectFiles: true`). Without selection, the file
wasn't marked as active, preventing tool access.

Additionally, `AppInitializer` was placed outside
`ToolWorkflowProvider`, causing a context error.

## Solution

### Changes:
1. **frontend/src/desktop/hooks/useAppInitialization.ts**
   - Added `{ selectFiles: true }` when calling `addFiles()`
   - Files now immediately marked as active in FileContext

2. **frontend/src/core/components/AppProviders.tsx**
   - Moved `AppInitializer` inside `ToolWorkflowProvider`
   - Ensures context availability for initialization

## Testing
- Open PDF via "Open with" on Windows
- File now immediately usable with all tools
- No manual reselection needed

## Screenshot
<img width="1920" height="1080" alt="Screenshot (3)"
src="https://github.com/user-attachments/assets/9ceacadf-eb12-42a6-86f9-bca6188bfbb9"
/>

---------

Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
Co-authored-by: EthanHealy01 <80844253+EthanHealy01@users.noreply.github.com>
Co-authored-by: Reece Browne <74901996+reecebrowne@users.noreply.github.com>
2026-04-08 20:39:20 +05:30
Reece 2306200c1e Improvements 2026-04-08 14:58:35 +01:00
Anthony Stirlinganda 81f20504ad pipeline fixes (#6068)
Co-authored-by: a <a>
2026-04-04 10:19:38 +01:00
Reece Browne 3ae0b88c23 Line seperator fix for redaction drift (#6064) 2026-04-03 17:47:48 +01:00
Anthony Stirling 7d79ed4148 possible fix permission issues and fix thread timing issues (#6061)
# Description of Changes

<!--
Please provide a summary of the changes, including:

- What was changed
- Why the change was made
- Any challenges encountered

Closes #(issue_number)
-->

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md#6-testing)
for more details.
2026-04-03 16:49:16 +01:00
EthanHealy01andReece Browne aeb7695617 Add specific View Scope For Selected Files (#6050)
## Fix 1 — Viewer bug (8 tools)

8 tools called `useFileSelection()` directly instead of routing through
`useBaseTool`. In the viewer, this meant they operated on **all selected
files**
instead of only the one being viewed. For example: 10 files loaded,
viewing
file 3, running Add Stamp — all 10 files got stamped.

**Root cause:** These tools had no view-scope awareness.
`useFileSelection()`
returns the raw workbench selection with no knowledge of which file is
active in
the viewer.

**Fix:** A new hook `useViewScopedFiles` was introduced:

```ts
// Viewer → only the active file
// Everywhere else → all loaded files
const selectedFiles = useViewScopedFiles();
```

The 8 tools were updated to call this instead of `useFileSelection()`.

**Tools fixed:** Add Stamp, Add Watermark, Add Password, Add Page
Numbers,
Add Attachments, Reorganize Pages, OCR, Convert

---

## Fix 2 — Page selector / active files context (all tools)

`useBaseTool` returned `selectedFiles` (checked files only) in
non-viewer
contexts. In the page selector this is typically empty or stale — not
the full
set of loaded files that tools should operate on.

**Fix:** `useBaseTool` was updated to use `useViewScopedFiles`, which
returns
all loaded files in non-viewer contexts. This affected every tool via
`useBaseTool`.

---

## Workarounds for Compare & Merge

Two tools intentionally need all loaded files regardless of view, so
they use
`ignoreViewerScope: true` in `useBaseTool`.

**Compare** — needs exactly 2 files for its Original/Edited slots.
Scoping to
one file would break the comparison entirely. `ignoreViewerScope: true`
is set
and `disableScopeHints: true` hides the "(this file)" button label hint.
The
slot auto-mapping logic was also improved alongside this fix.

**Merge** — needs 2+ files; merging a single file is meaningless. Rather
than
leaving the button silently disabled, Merge now:
- Auto-redirects to the active files view on first open from the viewer
- If the user navigates back to the viewer, shows a disabled button with
a hint
  and a "Go to active files view" shortcut button

---

## How to Test

---

## Fix 1 — 8 tools (viewer scoping)

### Test steps (same for each)
1. Load 3 PDFs into workbench
2. Open viewer, navigate to file 2
3. Open the tool, configure settings, run
4.  Only file 2 is in the results
5.  Button label shows **"[Action] (this file)"**
6.  A note below the button reads **"Only applying to: [filename]"**

| Tool | What to configure |
|---|---|
| **Add Stamp** | Enter any text stamp or upload an image stamp |
| **Add Watermark** | Select text watermark, enter any text |
| **Add Page Numbers** | Leave defaults |
| **Add Password** | Enter any owner + user password |
| **Add Attachments** | Attach any small file |
| **Reorganize Pages** | Enter a page range e.g. `1,2` |
| **OCR** | Leave default language |
| **Convert** | Convert PDF → any format |

---

## Fix 2 — All tools (page selector context)

### Test steps
1. Load 3 PDFs into workbench
2. Open the page selector view 
3. Open any tool from the sidebar, run it
4.  All 3 files are processed (not zero or a stale subset)

---

## Compare (intentionally ignores view scope)

**A — Auto-fill with exactly 2 files**
1. Load exactly 2 PDFs
2. Open Compare from either the viewer or active files view
3.  Both slots are filled automatically (Original + Edited)
4.  No scope hint appears on the button

**B — Manual selection with 3+ files**
1. Load 3+ PDFs
2. Open Compare
3.  The first 2 files fill the slots
4.  A 3rd file does not add a 3rd slot (capped at 2)

**C — File removed mid-session**
1. Load 2 PDFs, let Compare auto-fill both slots
2. Remove one file from the workbench
3.  The corresponding slot clears; the other slot is unchanged

**D — Viewer mode**
1. Load 2 PDFs, open viewer
2. Open Compare from the viewer sidebar
3.  Both files are still available for slot selection (not scoped to
current file)

---

## Merge (intentionally ignores view scope, disabled in viewer)

**A — Auto-redirect on first open from viewer**
1. Load 2+ PDFs, open the viewer
2. Open Merge from the viewer sidebar
3.  Immediately redirected to the active files view

**B — Viewer mode disabled state (after navigating back)**
1. From the active files view, open Merge, then navigate back to the
viewer
2.  Execute button is **disabled** with tooltip "Switch to the file
editor to select multiple files"
3.  A note appears: *"Merge needs 2 or more files. Head to the file
editor to select them."*
4.  A **"Go to active files view"** button is shown; clicking it
navigates back

**C — Active files view works normally**
1. Load 3 PDFs, open Merge from the active files view
2.  All 3 files appear in the merge list
3.  Button shows **"Merge (3 files)"**
4. Run the merge
5.  Output is a single PDF containing all 3 files

---

## Button label behaviour (all tools)

| Context | Expected button text |
|---|---|
| Viewer, 1 file loaded | `[Action]` (no suffix) |
| Viewer, 2+ files loaded | `[Action] (this file)` |
| Active files view, 1 file loaded | `[Action]` (no suffix) |
| Active files view, 2+ files loaded | `[Action] (N files)` |
| Merge in viewer | disabled — no suffix |
| Compare | never shows scope suffix (`disableScopeHints: true`) |

---------

Co-authored-by: Reece Browne <74901996+reecebrowne@users.noreply.github.com>
2026-04-03 16:04:38 +01:00
Anthony Stirling 69bd7cda96 dep updates (#6058) 2026-04-03 13:24:41 +01:00
stirlingbot[bot]andAnthony Stirling fd66648df7 🤖 format everything with pre-commit by stirlingbot (#6000)
Co-authored-by: stirlingbot[bot] <195170888+stirlingbot[bot]@users.noreply.github.com>
Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
2026-04-02 18:47:52 +01:00
Thomas BERNARD ec00124758 translate more messages to fr-FR (#6042) 2026-04-02 17:55:17 +01:00
Dexterity 2ab40423e2 Fix image stamp cropping and align preview with PDF output for add-stamp (#6013) 2026-04-02 17:54:13 +01:00
Anthony Stirling 36c0540b37 removeffmpeg (#6053) 2026-04-02 17:40:02 +01:00
Reece Browne 1d844f6c80 Fix/redact bug (#6048) 2026-04-02 17:39:45 +01:00
Anthony Stirlinganda 69eaa1c6b0 Pipeline changes and version bump (#6047)
# Description of Changes

<!--
Please provide a summary of the changes, including:

- What was changed
- Why the change was made
- Any challenges encountered

Closes #(issue_number)
-->

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md#6-testing)
for more details.

---------

Co-authored-by: a <a>
2026-04-02 12:52:22 +01:00
Peter Dave Hello 04e52ee06d Restore English search aliases in zh-TW tags (#6039)
# Description of Changes

Preserve the translated zh-TW tags while restoring the English aliases
used by frontend tool search.

This keeps common English technical queries such as permissions or
access control discoverable in the zh-TW locale.
---

## Checklist

### General

- [x] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [x] I have performed a self-review of my own code
- [x] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [x] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md#6-testing)
for more details.

## GitHub Copilot Pull Reuqest summary

> This pull request significantly expands the keyword tags for a wide
range of PDF-related tools and actions in the Traditional Chinese
(`zh-TW`) translation file. The main goal is to improve searchability
and discoverability of features by including a comprehensive set of
English and Chinese keywords, synonyms, and related phrases for each
tool.
> 
> The most important changes include:
> 
> **Localization and Search Optimization:**
> 
> * Expanded the `tags` fields for all tools and actions under the
`[home.*]` sections in `frontend/public/locales/zh-TW/translation.toml`
to include a broad set of English and Chinese keywords, synonyms, and
common search phrases. This enhances feature discoverability for users
searching in either language.
[[1]](diffhunk://#diff-5979ec7aabfd804ffe625390faee80bfc5b97bdb00f72cc3ce27359f82450e87L3885-R3925)
[[2]](diffhunk://#diff-5979ec7aabfd804ffe625390faee80bfc5b97bdb00f72cc3ce27359f82450e87L3934-R4039)
[[3]](diffhunk://#diff-5979ec7aabfd804ffe625390faee80bfc5b97bdb00f72cc3ce27359f82450e87L4054-R4209)
[[4]](diffhunk://#diff-5979ec7aabfd804ffe625390faee80bfc5b97bdb00f72cc3ce27359f82450e87L4218-R4218)
> 
> **Consistency and Coverage:**
> 
> * Ensured that each tool/action now has a rich set of tags that cover
various ways users might refer to the feature, including technical
terms, synonyms, and related concepts (e.g., "merge", "combine", "join"
for PDF merging).
[[1]](diffhunk://#diff-5979ec7aabfd804ffe625390faee80bfc5b97bdb00f72cc3ce27359f82450e87L3885-R3925)
[[2]](diffhunk://#diff-5979ec7aabfd804ffe625390faee80bfc5b97bdb00f72cc3ce27359f82450e87L3934-R4039)
[[3]](diffhunk://#diff-5979ec7aabfd804ffe625390faee80bfc5b97bdb00f72cc3ce27359f82450e87L4054-R4209)
[[4]](diffhunk://#diff-5979ec7aabfd804ffe625390faee80bfc5b97bdb00f72cc3ce27359f82450e87L4218-R4218)
> 
> **Internationalization Improvements:**
> 
> * Added English keywords alongside Chinese ones to support bilingual
search and better serve users who may search using English terms in a
localized interface.
[[1]](diffhunk://#diff-5979ec7aabfd804ffe625390faee80bfc5b97bdb00f72cc3ce27359f82450e87L3885-R3925)
[[2]](diffhunk://#diff-5979ec7aabfd804ffe625390faee80bfc5b97bdb00f72cc3ce27359f82450e87L3934-R4039)
[[3]](diffhunk://#diff-5979ec7aabfd804ffe625390faee80bfc5b97bdb00f72cc3ce27359f82450e87L4054-R4209)
[[4]](diffhunk://#diff-5979ec7aabfd804ffe625390faee80bfc5b97bdb00f72cc3ce27359f82450e87L4218-R4218)
> 
> These changes collectively make it easier for users to find the
features they need, regardless of the language or terminology they use.
2026-04-02 08:35:38 +00:00
EthanHealy01 2648058fd5 bump deps (#6041)
bump deps and add a one week buffer to releases that we merge in to
allow for vulnerabilities to be caught.
2026-04-01 18:08:45 +01:00
ConnorYoh 8e200658a0 Alpha flag for file storage settings (#6044)
## Summary
- Added "Alpha" badge to the File Storage & Sharing nav item in the
settings sidebar
- Added "Alpha" badge to the File Storage & Sharing page title
- Removed the old inline "(Alpha)" text from the Enable Group Signing
label
- Restructured all toggle cards so the switch is anchored to the right
of each row
- Tightened spacing between cards for a more compact layout
- Extended `ConfigNavItem` interface with optional `badge` and
`badgeColor` fields for reuse elsewhere
<img width="1696" height="1057" alt="image"
src="https://github.com/user-attachments/assets/77ac8276-ed65-4cae-8470-65de8f56dd74"
/>
2026-04-01 17:18:48 +01:00
EthanHealy01 9dcec10c06 Bug/connection mode fixes (#5998) 2026-04-01 15:33:46 +01:00
Anthony Stirling c8296af41c fix new line in redact (#6035) 2026-04-01 11:58:38 +01:00
Anthony Stirling bcb4f3b132 idle cpu fix test (#6015) 2026-04-01 11:58:10 +01:00
Anthony Stirling e78cf0564b qr split fixes (#6043) 2026-04-01 11:54:33 +01:00
Anthony Stirling 7058cc2a58 Remove gosu (#6036) 2026-04-01 11:54:12 +01:00
Matheus Saito d68e6b6a29 Added back ctrl+r as rotate if on desktop (#5982) (#5993)
Fix #5982

Behaviour of ctrl+r altered to support rotate on desktop, while the web
version continue to use refresh as default.
2026-04-01 11:48:53 +01:00
James Brunton 521629ec86 Fix any type usage in proprietary/ (#5949)
# Description of Changes
Follow on from #5934, expanding `any` type usage ban to the
`proprietary/` folder
2026-04-01 08:21:26 +00:00
1814 changed files with 153457 additions and 64942 deletions
+1
View File
@@ -65,6 +65,7 @@ README*
.env
.env.*
!.env.example
!engine/.env
# Misc
*.swp
+1
View File
@@ -15,6 +15,7 @@ max_line_length = 100
[*.py]
indent_size = 4
max_line_length = 120
[*.gradle]
indent_size = 4
+29
View File
@@ -0,0 +1,29 @@
# Maintainer: Stirling PDF Inc <contact@stirlingpdf.com>
pkgname=stirling-pdf-bin
pkgver=2.7.3
pkgrel=1
pkgdesc="Locally hosted, web-based PDF manipulation tool (desktop app, prebuilt binary)"
arch=('x86_64')
url="https://www.stirling.com"
license=('MIT' 'LicenseRef-Stirling-PDF-Proprietary')
depends=('gtk3' 'webkit2gtk' 'libappindicator-gtk3')
provides=('stirling-pdf')
conflicts=('stirling-pdf' 'stirling-pdf-git')
options=('!strip')
source_x86_64=("${pkgname}-${pkgver}.deb::https://github.com/Stirling-Tools/Stirling-PDF/releases/download/v${pkgver}/Stirling-PDF-linux-x86_64.deb")
sha256sums_x86_64=('PLACEHOLDER_DEB_SHA256')
package() {
# Extract the .deb archive
bsdtar -xf data.tar* -C "${pkgdir}"
# Fix permissions
find "${pkgdir}" -type d -exec chmod 755 {} \;
# Install license
install -Dm644 /dev/stdin "${pkgdir}/usr/share/licenses/${pkgname}/LICENSE" <<EOF
Copyright (c) 2025 Stirling PDF Inc
All rights reserved. See https://github.com/Stirling-Tools/Stirling-PDF/blob/main/LICENSE
EOF
}
@@ -0,0 +1,90 @@
# Maintainer: Stirling PDF Inc <contact@stirlingpdf.com>
pkgname=stirling-pdf-server-bin
pkgver=2.7.3
pkgrel=1
pkgdesc="Locally hosted, web-based PDF manipulation tool (server JAR, prebuilt)"
arch=('any')
url="https://www.stirling.com"
license=('MIT' 'LicenseRef-Stirling-PDF-Proprietary')
depends=('java-runtime>=21')
provides=('stirling-pdf-server')
conflicts=('stirling-pdf-server' 'stirling-pdf-server-git')
backup=('etc/stirling-pdf-server/settings.yml')
source=("Stirling-PDF-with-login-${pkgver}.jar::https://github.com/Stirling-Tools/Stirling-PDF/releases/download/v${pkgver}/Stirling-PDF-with-login.jar"
"stirling-pdf-server.service"
"stirling-pdf-server.sysusers"
"stirling-pdf-server.tmpfiles")
sha256sums=('PLACEHOLDER_JAR_SHA256'
'PLACEHOLDER_SERVICE_SHA256'
'PLACEHOLDER_SYSUSERS_SHA256'
'PLACEHOLDER_TMPFILES_SHA256')
prepare() {
cat > stirling-pdf-server.service << 'EOF'
[Unit]
Description=Stirling-PDF Server
After=network.target
[Service]
Type=simple
User=stirling-pdf
Group=stirling-pdf
WorkingDirectory=/var/lib/stirling-pdf-server
ExecStart=/usr/bin/java -jar /usr/share/stirling-pdf-server/stirling-pdf-server.jar
Restart=on-failure
RestartSec=5
StandardOutput=journal
StandardError=journal
SyslogIdentifier=stirling-pdf-server
Environment=JAVA_OPTS=-Xmx512m
[Install]
WantedBy=multi-user.target
EOF
cat > stirling-pdf-server.sysusers << 'EOF'
u stirling-pdf - "Stirling-PDF Server" /var/lib/stirling-pdf-server -
EOF
cat > stirling-pdf-server.tmpfiles << 'EOF'
d /var/lib/stirling-pdf-server 0750 stirling-pdf stirling-pdf -
d /var/log/stirling-pdf-server 0750 stirling-pdf stirling-pdf -
EOF
}
package() {
# JAR
install -Dm644 "Stirling-PDF-with-login-${pkgver}.jar" \
"${pkgdir}/usr/share/stirling-pdf-server/stirling-pdf-server.jar"
# Wrapper script
install -Dm755 /dev/stdin "${pkgdir}/usr/bin/stirling-pdf-server" << 'EOF'
#!/bin/sh
exec java $JAVA_OPTS -jar /usr/share/stirling-pdf-server/stirling-pdf-server.jar "$@"
EOF
# systemd unit
install -Dm644 stirling-pdf-server.service \
"${pkgdir}/usr/lib/systemd/system/stirling-pdf-server.service"
# sysusers / tmpfiles
install -Dm644 stirling-pdf-server.sysusers \
"${pkgdir}/usr/lib/sysusers.d/stirling-pdf-server.conf"
install -Dm644 stirling-pdf-server.tmpfiles \
"${pkgdir}/usr/lib/tmpfiles.d/stirling-pdf-server.conf"
# Default config stub
install -dm755 "${pkgdir}/etc/stirling-pdf-server"
install -Dm644 /dev/stdin "${pkgdir}/etc/stirling-pdf-server/settings.yml" << 'EOF'
# Stirling-PDF Server configuration
# See https://github.com/Stirling-Tools/Stirling-PDF for all options
server:
port: 8080
EOF
# License
install -Dm644 /dev/stdin "${pkgdir}/usr/share/licenses/${pkgname}/LICENSE" << 'EOF'
MIT License — see https://github.com/Stirling-Tools/Stirling-PDF/blob/main/LICENSE
EOF
}
+1 -2
View File
@@ -46,8 +46,7 @@ frontend: &frontend
- testing/**
- docker/**
- scripts/translations/*.py
- scripts/build-tauri-jlink.bat
- scripts/build-tauri-jlink.sh
- .taskfiles/desktop.yml
- scripts/convert_cff_to_ttf.py
- scripts/harvest_type3_fonts.py
- scripts/ignore_translation.toml
+3 -2
View File
@@ -17,7 +17,7 @@ Closes #(issue_number)
### General
- [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md) (if applicable)
- [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable)
- [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings
@@ -37,4 +37,5 @@ Closes #(issue_number)
### Testing (if applicable)
- [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md#6-testing) for more details.
- [ ] I have run `task check` to verify linters, typechecks, and tests pass
- [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details.
+145 -43
View File
@@ -3,6 +3,31 @@ name: PR Deployment via Comment
on:
issue_comment:
types: [created]
workflow_dispatch:
inputs:
pr_number:
description: "PR number to deploy"
required: true
enable_prototypes:
description: "Build with prototypes frontend"
required: false
type: boolean
default: false
enable_pro:
description: "Enable pro features"
required: false
type: boolean
default: false
enable_enterprise:
description: "Enable enterprise features"
required: false
type: boolean
default: false
disable_security:
description: "Disable security/login"
required: false
type: boolean
default: true
permissions:
contents: read
@@ -14,23 +39,27 @@ jobs:
permissions:
issues: write
if: |
vars.CI_PROFILE != 'lite' &&
github.event.issue.pull_request &&
(
contains(github.event.comment.body, 'prdeploy') ||
contains(github.event.comment.body, 'deploypr')
)
&&
(
github.event.comment.user.login == 'frooodle' ||
github.event.comment.user.login == 'sf298' ||
github.event.comment.user.login == 'Ludy87' ||
github.event.comment.user.login == 'balazs-szucs' ||
github.event.comment.user.login == 'reecebrowne' ||
github.event.comment.user.login == 'DarioGii' ||
github.event.comment.user.login == 'EthanHealy01' ||
github.event.comment.user.login == 'jbrunton96' ||
github.event.comment.user.login == 'ConnorYoh'
vars.CI_PROFILE != 'lite' && (
github.event_name == 'workflow_dispatch' ||
(
github.event.issue.pull_request &&
(
contains(github.event.comment.body, 'prdeploy') ||
contains(github.event.comment.body, 'deploypr')
)
&&
(
github.event.comment.user.login == 'frooodle' ||
github.event.comment.user.login == 'sf298' ||
github.event.comment.user.login == 'Ludy87' ||
github.event.comment.user.login == 'balazs-szucs' ||
github.event.comment.user.login == 'reecebrowne' ||
github.event.comment.user.login == 'DarioGii' ||
github.event.comment.user.login == 'EthanHealy01' ||
github.event.comment.user.login == 'jbrunton96' ||
github.event.comment.user.login == 'ConnorYoh'
)
)
)
outputs:
pr_number: ${{ steps.get-pr.outputs.pr_number }}
@@ -38,6 +67,7 @@ jobs:
disable_security: ${{ steps.check-security-flag.outputs.disable_security }}
enable_pro: ${{ steps.check-pro-flag.outputs.enable_pro }}
enable_enterprise: ${{ steps.check-pro-flag.outputs.enable_enterprise }}
enable_prototypes: ${{ steps.check-prototypes-flag.outputs.enable_prototypes }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@58077d3c7e43986b6b15fba718e8ea69e387dfcc # v2.15.1
@@ -61,7 +91,9 @@ jobs:
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
with:
script: |
const prNumber = context.payload.issue.number;
const prNumber = context.eventName === 'workflow_dispatch'
? context.payload.inputs.pr_number
: context.payload.issue.number;
console.log(`PR Number: ${prNumber}`);
core.setOutput('pr_number', prNumber);
@@ -69,12 +101,14 @@ jobs:
id: check-security-flag
env:
COMMENT_BODY: ${{ github.event.comment.body }}
IS_DISPATCH: ${{ github.event_name == 'workflow_dispatch' }}
DISPATCH_DISABLE_SECURITY: ${{ inputs.disable_security }}
run: |
if [[ "$COMMENT_BODY" == *"security"* ]] || [[ "$COMMENT_BODY" == *"login"* ]]; then
echo "Security flags detected in comment"
if [[ "$IS_DISPATCH" == "true" ]]; then
echo "disable_security=$DISPATCH_DISABLE_SECURITY" >> $GITHUB_OUTPUT
elif [[ "$COMMENT_BODY" == *"security"* ]] || [[ "$COMMENT_BODY" == *"login"* ]]; then
echo "disable_security=false" >> $GITHUB_OUTPUT
else
echo "No security flags detected in comment"
echo "disable_security=true" >> $GITHUB_OUTPUT
fi
@@ -82,22 +116,43 @@ jobs:
id: check-pro-flag
env:
COMMENT_BODY: ${{ github.event.comment.body }}
IS_DISPATCH: ${{ github.event_name == 'workflow_dispatch' }}
DISPATCH_PRO: ${{ inputs.enable_pro }}
DISPATCH_ENTERPRISE: ${{ inputs.enable_enterprise }}
run: |
if [[ "$COMMENT_BODY" == *"pro"* ]] || [[ "$COMMENT_BODY" == *"premium"* ]]; then
echo "pro flags detected in comment"
if [[ "$IS_DISPATCH" == "true" ]]; then
echo "enable_pro=$DISPATCH_PRO" >> $GITHUB_OUTPUT
echo "enable_enterprise=$DISPATCH_ENTERPRISE" >> $GITHUB_OUTPUT
elif [[ "$COMMENT_BODY" == *"pro"* ]] || [[ "$COMMENT_BODY" == *"premium"* ]]; then
echo "enable_pro=true" >> $GITHUB_OUTPUT
echo "enable_enterprise=false" >> $GITHUB_OUTPUT
elif [[ "$COMMENT_BODY" == *"enterprise"* ]]; then
echo "enterprise flags detected in comment"
echo "enable_enterprise=true" >> $GITHUB_OUTPUT
echo "enable_pro=true" >> $GITHUB_OUTPUT
else
echo "No pro or enterprise flags detected in comment"
echo "enable_pro=false" >> $GITHUB_OUTPUT
echo "enable_enterprise=false" >> $GITHUB_OUTPUT
fi
- name: Check for prototypes flag
id: check-prototypes-flag
env:
COMMENT_BODY: ${{ github.event.comment.body }}
IS_DISPATCH: ${{ github.event_name == 'workflow_dispatch' }}
DISPATCH_PROTOTYPES: ${{ inputs.enable_prototypes }}
run: |
if [[ "$IS_DISPATCH" == "true" ]]; then
echo "enable_prototypes=$DISPATCH_PROTOTYPES" >> $GITHUB_OUTPUT
elif [[ "$COMMENT_BODY" == *"prototypes"* ]]; then
echo "Prototypes flag detected in comment"
echo "enable_prototypes=true" >> $GITHUB_OUTPUT
else
echo "No prototypes flag detected in comment"
echo "enable_prototypes=false" >> $GITHUB_OUTPUT
fi
- name: Add 'in_progress' reaction to comment
if: github.event_name == 'issue_comment'
id: add-eyes-reaction
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
with:
@@ -161,6 +216,8 @@ jobs:
with:
gradle-version: 9.3.1
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
- name: Run Gradle Command
run: |
if [ "${{ needs.check-comment.outputs.disable_security }}" == "true" ]; then
@@ -168,7 +225,7 @@ jobs:
else
export DISABLE_ADDITIONAL_FEATURES=false
fi
./gradlew build
task backend:build
env:
MAVEN_USER: ${{ secrets.MAVEN_USER }}
MAVEN_PASSWORD: ${{ secrets.MAVEN_PASSWORD }}
@@ -193,7 +250,21 @@ jobs:
cache-from: type=gha,scope=stirling-pdf-latest
cache-to: type=gha,mode=max,scope=stirling-pdf-latest
tags: ${{ secrets.DOCKER_HUB_USERNAME }}/test:pr-${{ needs.check-comment.outputs.pr_number }}
build-args: VERSION_TAG=alpha
build-args: |
VERSION_TAG=alpha
PROTOTYPES_BUILD=${{ needs.check-comment.outputs.enable_prototypes }}
platforms: linux/amd64
- name: Build and push engine image
if: needs.check-comment.outputs.enable_prototypes == 'true'
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: ./engine
file: ./engine/Dockerfile
push: true
cache-from: type=gha,scope=stirling-pdf-engine
cache-to: type=gha,mode=max,scope=stirling-pdf-engine
tags: ${{ secrets.DOCKER_HUB_USERNAME }}/test:engine-pr-${{ needs.check-comment.outputs.pr_number }}
platforms: linux/amd64
- name: Set up SSH
@@ -231,33 +302,64 @@ jobs:
PREMIUM_PROFEATURES_AUDIT_ENABLED="false"
fi
ENABLE_PROTOTYPES="${{ needs.check-comment.outputs.enable_prototypes }}"
PR_NUMBER="${{ needs.check-comment.outputs.pr_number }}"
DOCKER_USER="${{ secrets.DOCKER_HUB_USERNAME }}"
# Build engine env vars for backend (only set when prototypes enabled)
if [ "$ENABLE_PROTOTYPES" == "true" ]; then
AI_ENGINE_VARS="
SYSTEM_AIENGINE_ENABLED: \"true\"
SYSTEM_AIENGINE_URL: \"http://stirling-pdf-engine-pr-${PR_NUMBER}:5001\""
ENGINE_SERVICE="
stirling-pdf-engine:
container_name: stirling-pdf-engine-pr-${PR_NUMBER}
image: ${DOCKER_USER}/test:engine-pr-${PR_NUMBER}
environment:
ANTHROPIC_API_KEY: \"${{ secrets.ANTHROPIC_API_KEY }}\"
networks:
- pr-network
restart: on-failure:5"
NETWORK_SECTION="
networks:
pr-network:"
BACKEND_NETWORK="
networks:
- pr-network"
else
AI_ENGINE_VARS=""
ENGINE_SERVICE=""
NETWORK_SECTION=""
BACKEND_NETWORK=""
fi
# First create the docker-compose content locally
cat > docker-compose.yml << EOF
version: '3.3'
services:
stirling-pdf:
container_name: stirling-pdf-pr-${{ needs.check-comment.outputs.pr_number }}
image: ${{ secrets.DOCKER_HUB_USERNAME }}/test:pr-${{ needs.check-comment.outputs.pr_number }}
container_name: stirling-pdf-pr-${PR_NUMBER}
image: ${DOCKER_USER}/test:pr-${PR_NUMBER}
ports:
- "${{ needs.check-comment.outputs.pr_number }}:8080"
- "${PR_NUMBER}:8080"
volumes:
- /stirling/PR-${{ needs.check-comment.outputs.pr_number }}/data:/usr/share/tessdata:rw
- /stirling/PR-${{ needs.check-comment.outputs.pr_number }}/config:/configs:rw
- /stirling/PR-${{ needs.check-comment.outputs.pr_number }}/logs:/logs:rw
- /stirling/PR-${PR_NUMBER}/data:/usr/share/tessdata:rw
- /stirling/PR-${PR_NUMBER}/config:/configs:rw
- /stirling/PR-${PR_NUMBER}/logs:/logs:rw
environment:
DISABLE_ADDITIONAL_FEATURES: "${DISABLE_ADDITIONAL_FEATURES}"
SECURITY_ENABLELOGIN: "${LOGIN_SECURITY}"
SYSTEM_DEFAULTLOCALE: en-GB
UI_APPNAME: "Stirling-PDF PR#${{ needs.check-comment.outputs.pr_number }}"
UI_HOMEDESCRIPTION: "PR#${{ needs.check-comment.outputs.pr_number }} for Stirling-PDF Latest"
UI_APPNAMENAVBAR: "PR#${{ needs.check-comment.outputs.pr_number }}"
UI_APPNAME: "Stirling-PDF PR#${PR_NUMBER}"
UI_HOMEDESCRIPTION: "PR#${PR_NUMBER} for Stirling-PDF Latest"
UI_APPNAMENAVBAR: "PR#${PR_NUMBER}"
SYSTEM_MAXFILESIZE: "100"
METRICS_ENABLED: "true"
SYSTEM_GOOGLEVISIBILITY: "false"
PREMIUM_KEY: "${PREMIUM_KEY}"
PREMIUM_ENABLED: "${PREMIUM_ENABLED}"
PREMIUM_PROFEATURES_AUDIT_ENABLED: "${PREMIUM_PROFEATURES_AUDIT_ENABLED}"
restart: on-failure:5
PREMIUM_PROFEATURES_AUDIT_ENABLED: "${PREMIUM_PROFEATURES_AUDIT_ENABLED}"${AI_ENGINE_VARS}
restart: on-failure:5${BACKEND_NETWORK}${ENGINE_SERVICE}${NETWORK_SECTION}
EOF
# Then copy the file and execute commands
@@ -265,13 +367,13 @@ jobs:
ssh -i ../private.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -T ${{ secrets.NEW_VPS_USERNAME }}@${{ secrets.NEW_VPS_HOST }} << ENDSSH
# Create PR-specific directories
mkdir -p /stirling/PR-${{ needs.check-comment.outputs.pr_number }}/{data,config,logs}
mkdir -p /stirling/PR-${PR_NUMBER}/{data,config,logs}
# Move docker-compose file to correct location
mv /tmp/docker-compose.yml /stirling/PR-${{ needs.check-comment.outputs.pr_number }}/docker-compose.yml
mv /tmp/docker-compose.yml /stirling/PR-${PR_NUMBER}/docker-compose.yml
# Start or restart the container
cd /stirling/PR-${{ needs.check-comment.outputs.pr_number }}
cd /stirling/PR-${PR_NUMBER}
docker-compose pull
docker-compose up -d
ENDSSH
@@ -280,7 +382,7 @@ jobs:
echo "security_status=${SECURITY_STATUS}" >> $GITHUB_ENV
- name: Add success reaction to comment
if: success()
if: success() && github.event_name == 'issue_comment'
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
with:
github-token: ${{ steps.setup-bot.outputs.token }}
@@ -315,7 +417,7 @@ jobs:
}
- name: Add failure reaction to comment
if: failure()
if: failure() && github.event_name == 'issue_comment'
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
with:
github-token: ${{ steps.setup-bot.outputs.token }}
+2 -1
View File
@@ -121,8 +121,9 @@ jobs:
# Remove PR-specific directories
rm -rf /stirling/PR-${{ github.event.pull_request.number }}
# Remove the Docker image
# Remove the Docker images
docker rmi --no-prune ${{ secrets.DOCKER_HUB_USERNAME }}/test:pr-${{ github.event.pull_request.number }} || true
docker rmi --no-prune ${{ secrets.DOCKER_HUB_USERNAME }}/test:engine-pr-${{ github.event.pull_request.number }} || true
echo "PERFORMED_CLEANUP"
else
+69 -48
View File
@@ -11,10 +11,6 @@ jobs:
permissions:
contents: read
pull-requests: write
defaults:
run:
working-directory: engine
steps:
- name: Checkout code
uses: actions/checkout@v4
@@ -24,60 +20,85 @@ jobs:
with:
enable-cache: true
- name: Install dependencies
run: make install
- name: Run fixers
# Ignore errors here because we're going to add comments for them in the following steps before actually failing
run: make fix || true
- name: Check for fixer changes
id: fixer_changes
run: |
if git diff --quiet; then
echo "changed=false" >> "$GITHUB_OUTPUT"
else
echo "changed=true" >> "$GITHUB_OUTPUT"
fi
- name: Post fixer suggestions
if: steps.fixer_changes.outputs.changed == 'true' && github.event_name == 'pull_request'
uses: reviewdog/action-suggester@v1
continue-on-error: true
- name: Set up JDK 25
uses: actions/setup-java@be666c2fcd27ec809703dec50e508c2fdc7f6654 # v5.2.0
with:
tool_name: engine-make-fix
github_token: ${{ secrets.GITHUB_TOKEN }}
filter_mode: file
fail_level: any
level: info
java-version: "25"
distribution: "temurin"
- name: Comment on fixer suggestions
if: steps.fixer_changes.outputs.changed == 'true' && github.event_name == 'pull_request'
uses: actions/github-script@v7
- name: Setup Gradle
uses: gradle/actions/setup-gradle@f29f5a9d7b09a7c6b29859002d29d24e1674c884 # v5.0.1
with:
script: |
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
body: "The Python code in your PR has formatting/linting issues. Consider running `make fix` locally or setting up your editor's Ruff integration to auto-format and lint your files as you go, or commit the suggested changes on this PR.",
});
gradle-version: 9.3.1
- name: Verify fixer changes are committed
if: steps.fixer_changes.outputs.changed == 'true'
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
- name: Regenerate tool models
run: task engine:tool-models
- name: Verify tool models are up to date
run: |
if ! git diff --exit-code; then
echo "Fixes are out of date."
echo "Apply the reviewdog suggestions or run 'make fix' from engine/ and commit the updated files."
git --no-pager diff --stat
if ! git diff --exit-code engine/src/stirling/models/tool_models.py; then
echo "tool_models.py is out of date."
echo "Run 'task engine:tool-models' locally and commit the updated file."
exit 1
fi
- name: Run fixers
run: task engine:fix
- name: Verify fixes are committed
id: fixer_changes
run: |
if ! git diff --quiet; then
git --no-pager diff --stat
echo "::error::There are issues with your Python code that will need to be fixed before they can be merged in. Run 'task engine:fix' to auto-fix what can be fixed automatically, then run 'task engine:check' to see what still needs fixing manually."
exit 1
fi
- name: Comment on fixer failures
if: steps.fixer_changes.outcome == 'failure' && github.event_name == 'pull_request'
continue-on-error: true
uses: actions/github-script@v7
with:
script: |
const marker = '<!-- engine-check -->';
const body = [
marker,
'### Engine Check Failed',
'',
'There are issues with your Python code that will need to be fixed before they can be merged in.',
'',
'Run `task engine:fix` to auto-fix what can be fixed automatically, then run `task engine:check` to see what still needs fixing manually.',
].join('\n');
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
});
const existing = comments.find(c => c.body.includes(marker));
if (existing) {
await github.rest.issues.updateComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: existing.id,
body,
});
} else {
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
body,
});
}
- name: Run linting
run: make lint
run: task engine:lint
- name: Run type checking
run: make typecheck
run: task engine:typecheck
- name: Run tests
run: make test
run: task engine:test
+128
View File
@@ -0,0 +1,128 @@
name: Publish to AUR
on:
release:
types: [released]
workflow_dispatch:
inputs:
version:
description: "Version to publish (e.g. 2.9.2 — no v prefix)"
required: true
type: string
dry_run:
description: "Skip the AUR push (safe test)"
type: boolean
default: true
permissions:
contents: read
jobs:
get-release-info:
runs-on: ubuntu-latest
outputs:
version: ${{ steps.info.outputs.version }}
deb_sha256: ${{ steps.hashes.outputs.deb_sha256 }}
jar_sha256: ${{ steps.hashes.outputs.jar_sha256 }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@6c3c2f2c1c457b00c10c4848d6f5491db3b629df # v2.18.0
with:
egress-policy: audit
- name: Extract version from tag or manual input
id: info
env:
DISPATCH_VERSION: ${{ inputs.version }}
RELEASE_TAG: ${{ github.event.release.tag_name }}
run: |
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
VERSION="$DISPATCH_VERSION"
else
VERSION="$RELEASE_TAG"
fi
VERSION="${VERSION#v}"
echo "version=$VERSION" >> "$GITHUB_OUTPUT"
- name: Download release assets and compute SHA256
id: hashes
env:
VERSION: ${{ steps.info.outputs.version }}
run: |
BASE="https://github.com/Stirling-Tools/Stirling-PDF/releases/download/v${VERSION}"
download_sha256() {
local url="$1"
local file
file=$(basename "$url")
curl -fsSL --retry 3 -o "$file" "$url"
sha256sum "$file" | awk '{print $1}'
}
DEB_SHA=$(download_sha256 "${BASE}/Stirling-PDF-linux-x86_64.deb")
JAR_SHA=$(download_sha256 "${BASE}/Stirling-PDF-with-login.jar")
echo "deb_sha256=$DEB_SHA" >> "$GITHUB_OUTPUT"
echo "jar_sha256=$JAR_SHA" >> "$GITHUB_OUTPUT"
publish-aur:
needs: get-release-info
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@v2
with:
egress-policy: audit
- name: Checkout repository (for PKGBUILD templates)
uses: actions/checkout@v4
- name: Update stirling-pdf-bin PKGBUILD
env:
VERSION: ${{ needs.get-release-info.outputs.version }}
DEB_SHA: ${{ needs.get-release-info.outputs.deb_sha256 }}
run: |
PKGBUILD=".github/aur/stirling-pdf-bin/PKGBUILD"
sed -i "s/^pkgver=.*/pkgver=${VERSION}/" "$PKGBUILD"
sed -i "s/^pkgrel=.*/pkgrel=1/" "$PKGBUILD"
sed -i "s/'PLACEHOLDER_DEB_SHA256'/'${DEB_SHA}'/" "$PKGBUILD"
- name: Update stirling-pdf-server-bin PKGBUILD
env:
VERSION: ${{ needs.get-release-info.outputs.version }}
JAR_SHA: ${{ needs.get-release-info.outputs.jar_sha256 }}
run: |
PKGBUILD=".github/aur/stirling-pdf-server-bin/PKGBUILD"
sed -i "s/^pkgver=.*/pkgver=${VERSION}/" "$PKGBUILD"
sed -i "s/^pkgrel=.*/pkgrel=1/" "$PKGBUILD"
sed -i "s/'PLACEHOLDER_JAR_SHA256'/'${JAR_SHA}'/" "$PKGBUILD"
- name: Show updated PKGBUILDs (for dry-run visibility)
run: |
echo "--- stirling-pdf-bin PKGBUILD ---"
cat .github/aur/stirling-pdf-bin/PKGBUILD
echo ""
echo "--- stirling-pdf-server-bin PKGBUILD ---"
cat .github/aur/stirling-pdf-server-bin/PKGBUILD
- name: Publish stirling-pdf-bin to AUR
if: ${{ github.event_name == 'release' || inputs.dry_run == false }}
uses: KSXGitHub/github-actions-deploy-aur@2ac5a4c1d7035885d46b10e3193393be8460b6f1 # v4.1.1
with:
pkgname: stirling-pdf-bin
pkgbuild: .github/aur/stirling-pdf-bin/PKGBUILD
commit_username: Stirling PDF Inc
commit_email: contact@stirlingpdf.com
ssh_private_key: ${{ secrets.AUR_SSH_PRIVATE_KEY }}
commit_message: "Update to v${{ needs.get-release-info.outputs.version }}"
- name: Publish stirling-pdf-server-bin to AUR
if: ${{ github.event_name == 'release' || inputs.dry_run == false }}
uses: KSXGitHub/github-actions-deploy-aur@v4.1.1
with:
pkgname: stirling-pdf-server-bin
pkgbuild: .github/aur/stirling-pdf-server-bin/PKGBUILD
commit_username: Stirling PDF Inc
commit_email: contact@stirlingpdf.com
ssh_private_key: ${{ secrets.AUR_SSH_PRIVATE_KEY }}
commit_message: "Update to v${{ needs.get-release-info.outputs.version }}"
+146 -27
View File
@@ -50,6 +50,7 @@ jobs:
permissions:
actions: read
security-events: write
pull-requests: write
strategy:
fail-fast: false
matrix:
@@ -84,8 +85,77 @@ jobs:
gradle-version: 9.3.1
cache-disabled: true
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
- name: Check Java formatting (Spotless)
if: matrix.jdk-version == 25 && matrix.spring-security == false
id: spotless-check
run: task backend:format:check
continue-on-error: true
env:
MAVEN_USER: ${{ secrets.MAVEN_USER }}
MAVEN_PASSWORD: ${{ secrets.MAVEN_PASSWORD }}
MAVEN_PUBLIC_URL: ${{ secrets.MAVEN_PUBLIC_URL }}
- name: Comment on Java formatting failure
if: steps.spotless-check.outcome == 'failure'
continue-on-error: true
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
with:
script: |
const marker = '<!-- java-formatting-check -->';
const body = [
marker,
'### Java Formatting Check Failed',
'',
'Your code has formatting issues. Run the following command to fix them:',
'',
'```bash',
'task backend:format',
'```',
'',
'Then commit and push the changes.',
].join('\n');
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
});
const existing = comments.find(c => c.body.includes(marker));
if (existing) {
await github.rest.issues.updateComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: existing.id,
body,
});
} else {
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
body,
});
}
- name: Fail if Java formatting issues found
if: steps.spotless-check.outcome == 'failure'
run: |
echo "============================================"
echo " Java Formatting Check Failed"
echo "============================================"
echo ""
echo "Your code has formatting issues."
echo "Run the following command to fix them:"
echo ""
echo " task backend:format"
echo ""
echo "Then commit and push the changes."
echo "============================================"
exit 1
- name: Build with Gradle and spring security ${{ matrix.spring-security }}
run: ./gradlew build -PnoSpotless
run: task backend:build:ci
env:
MAVEN_USER: ${{ secrets.MAVEN_USER }}
MAVEN_PASSWORD: ${{ secrets.MAVEN_PASSWORD }}
@@ -169,8 +239,10 @@ jobs:
gradle-version: 9.3.1
cache-disabled: true
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
- name: Generate OpenAPI documentation
run: ./gradlew :stirling-pdf:generateOpenApiDocs
run: task backend:swagger
env:
MAVEN_USER: ${{ secrets.MAVEN_USER }}
MAVEN_PASSWORD: ${{ secrets.MAVEN_PASSWORD }}
@@ -187,6 +259,9 @@ jobs:
if: needs.files-changed.outputs.frontend == 'true'
needs: files-changed
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@58077d3c7e43986b6b15fba718e8ea69e387dfcc # v2.15.1
@@ -200,16 +275,63 @@ jobs:
node-version: "22"
cache: "npm"
cache-dependency-path: frontend/package-lock.json
- name: Install frontend dependencies
run: cd frontend && npm ci
- name: Type-check frontend
run: cd frontend && npm run prep && npm run typecheck:all
- name: Lint frontend
run: cd frontend && npm run lint
- name: Build frontend
run: cd frontend && npm run build
- name: Run frontend tests
run: cd frontend && npm run test -- --run
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
- name: Quality-check frontend
id: frontend-check
run: task frontend:check:all
continue-on-error: true
- name: Comment on frontend check failure
if: steps.frontend-check.outcome == 'failure'
continue-on-error: true
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
with:
script: |
const marker = '<!-- frontend-check -->';
const body = [
marker,
'### Frontend Check Failed',
'',
'There are issues with your frontend code that will need to be fixed before they can be merged in.',
'',
'Run `task frontend:fix` to auto-fix what can be fixed automatically, then run `task frontend:check:all` to see what still needs fixing manually.',
].join('\n');
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
});
const existing = comments.find(c => c.body.includes(marker));
if (existing) {
await github.rest.issues.updateComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: existing.id,
body,
});
} else {
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
body,
});
}
- name: Fail if frontend check failed
if: steps.frontend-check.outcome == 'failure'
run: |
echo "============================================"
echo " Frontend Check Failed"
echo "============================================"
echo ""
echo "There are issues with your frontend code that"
echo "will need to be fixed before they can be merged in."
echo ""
echo "Run 'task frontend:fix' to auto-fix what can be"
echo "fixed automatically, then run 'task frontend:check:all'"
echo "to see what still needs fixing manually."
echo "============================================"
exit 1
- name: Upload frontend build artifacts
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with:
@@ -234,14 +356,12 @@ jobs:
node-version: "22"
cache: "npm"
cache-dependency-path: frontend/package-lock.json
- name: Install frontend dependencies
run: cd frontend && npm ci
- name: Generate icons
run: cd frontend && node scripts/generate-icons.js
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
- name: Install Playwright (chromium only)
run: cd frontend && npx playwright install chromium --with-deps
run: task frontend:test:e2e:install -- chromium
- name: Run E2E tests (chromium)
run: cd frontend && npx playwright test --project=chromium
run: task frontend:test:e2e -- --project=chromium
- name: Upload Playwright report
if: always()
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
@@ -284,13 +404,10 @@ jobs:
gradle-version: 9.3.1
cache-disabled: true
- name: check the licenses for compatibility
# NOTE: --no-parallel is intentional here. Running the checkLicense task in parallel with other
# Gradle tasks has been observed to cause intermittent failures with the dependency license
# checking plugin on this Gradle version. Disabling parallel execution trades some build speed
# for more reliable, deterministic license checks. If upgrading Gradle or the plugin, consider
# re-evaluating whether this flag is still required before removing it.
run: ./gradlew checkLicense --no-parallel
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
- name: Check licenses for compatibility
run: task backend:licenses:check
env:
MAVEN_USER: ${{ secrets.MAVEN_USER }}
MAVEN_PASSWORD: ${{ secrets.MAVEN_PASSWORD }}
@@ -485,8 +602,10 @@ jobs:
gradle-version: 9.3.1
cache-disabled: true
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
- name: Build application
run: ./gradlew build
run: task backend:build
env:
MAVEN_USER: ${{ secrets.MAVEN_USER }}
MAVEN_PASSWORD: ${{ secrets.MAVEN_PASSWORD }}
@@ -513,7 +632,7 @@ jobs:
echo "base_image=stirling-pdf-base:pr-test" >> $GITHUB_OUTPUT
echo "platforms=linux/amd64" >> $GITHUB_OUTPUT
else
echo "base_image=ghcr.io/${{ steps.repoowner.outputs.lowercase }}/stirling-pdf-base:latest" >> $GITHUB_OUTPUT
echo "base_image=stirlingtools/stirling-pdf-base:latest" >> $GITHUB_OUTPUT
echo "platforms=linux/amd64,linux/arm64/v8" >> $GITHUB_OUTPUT
fi
@@ -89,12 +89,13 @@ jobs:
NPM_CONFIG_IGNORE_SCRIPTS: "true"
run: npm ci --ignore-scripts --audit=false --fund=false
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
- name: Generate frontend license report (internal PR)
if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork == false
working-directory: frontend
env:
PR_IS_FORK: "false"
run: npm run generate-licenses
run: task frontend:licenses:generate
- name: Generate frontend license report (fork PRs, pinned)
if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork == true
@@ -341,15 +342,11 @@ jobs:
with:
gradle-version: 9.3.1
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
- name: Check licenses and generate report
id: license-check
run: |
# NOTE: --no-parallel is intentional here. Running the license-checking tasks in parallel has
# previously caused intermittent concurrency issues in CI (e.g. flaky failures in the license
# plugin/Gradle when multiple projects are evaluated concurrently). Disabling parallelism trades
# some build speed for more reliable license reports. If the underlying issues are resolved in
# future Gradle or plugin versions, this flag can be reconsidered.
./gradlew checkLicense generateLicenseReport --no-parallel || echo "LICENSE_CHECK_FAILED=true" >> $GITHUB_ENV
run: task backend:licenses:generate || echo "LICENSE_CHECK_FAILED=true" >> $GITHUB_ENV
env:
MAVEN_USER: ${{ secrets.MAVEN_USER }}
MAVEN_PASSWORD: ${{ secrets.MAVEN_PASSWORD }}
+190 -166
View File
@@ -21,6 +21,14 @@ on:
- windows
- macos
- linux
sign:
description: "Code sign the binaries (requires signing secrets)"
required: false
default: "true"
type: choice
options:
- "true"
- "false"
release:
types: [created]
@@ -63,11 +71,11 @@ jobs:
with:
gradle-version: 9.3.1
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
- name: Get version number
id: versionNumber
run: |
echo "Running gradlew printVersion..."
./gradlew printVersion --quiet
VERSION=$(./gradlew printVersion --quiet | tail -1)
echo "Extracted version: $VERSION"
echo "versionNumber=$VERSION" >> $GITHUB_OUTPUT
@@ -144,6 +152,9 @@ jobs:
cache: "npm"
cache-dependency-path: frontend/package-lock.json
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
- name: Build JAR
run: ./gradlew build ${{ matrix.variant.build_frontend && '-PbuildWithFrontend=true' || '' }} -x spotlessApply -x spotlessCheck -x test -x sonarqube
env:
@@ -219,89 +230,21 @@ jobs:
with:
gradle-version: 9.3.1
- name: Build Java backend with JLink
working-directory: ./
shell: bash
run: |
chmod +x ./gradlew
echo "🔧 Building Stirling-PDF JAR..."
./gradlew build -x spotlessApply -x spotlessCheck -x test -x sonarqube
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
# Find the built JAR
STIRLING_JAR=$(ls app/core/build/libs/stirling-pdf-*.jar | head -n 1)
echo "✅ Built JAR: $STIRLING_JAR"
# Create Tauri directories
mkdir -p ./frontend/src-tauri/libs
mkdir -p ./frontend/src-tauri/runtime
# Copy JAR to Tauri libs
cp "$STIRLING_JAR" ./frontend/src-tauri/libs/
echo "✅ JAR copied to Tauri libs"
# Analyze JAR dependencies for jlink modules
echo "🔍 Analyzing JAR dependencies..."
if command -v jdeps &> /dev/null; then
DETECTED_MODULES=$(jdeps --print-module-deps --ignore-missing-deps "$STIRLING_JAR" 2>/dev/null || echo "")
if [ -n "$DETECTED_MODULES" ]; then
echo "📋 jdeps detected modules: $DETECTED_MODULES"
MODULES="$DETECTED_MODULES,java.compiler,java.instrument,java.management,java.naming,java.net.http,java.prefs,java.rmi,java.scripting,java.security.jgss,java.security.sasl,java.sql,java.transaction.xa,java.xml.crypto,jdk.crypto.ec,jdk.crypto.cryptoki,jdk.unsupported"
else
echo "⚠️ jdeps analysis failed, using predefined modules"
MODULES="java.base,java.compiler,java.desktop,java.instrument,java.logging,java.management,java.naming,java.net.http,java.prefs,java.rmi,java.scripting,java.security.jgss,java.security.sasl,java.sql,java.transaction.xa,java.xml,java.xml.crypto,jdk.crypto.ec,jdk.crypto.cryptoki,jdk.unsupported"
fi
else
echo "⚠️ jdeps not available, using predefined modules"
MODULES="java.base,java.compiler,java.desktop,java.instrument,java.logging,java.management,java.naming,java.net.http,java.prefs,java.rmi,java.scripting,java.security.jgss,java.security.sasl,java.sql,java.transaction.xa,java.xml,java.xml.crypto,jdk.crypto.ec,jdk.crypto.cryptoki,jdk.unsupported"
fi
# Create custom JRE with jlink
echo "🔧 Creating custom JRE with jlink..."
echo "📋 Using modules: $MODULES"
# Remove any existing JRE
rm -rf ./frontend/src-tauri/runtime/jre
# Create the custom JRE
jlink \
--add-modules "$MODULES" \
--strip-debug \
--compress=2 \
--no-header-files \
--no-man-pages \
--output ./frontend/src-tauri/runtime/jre
if [ ! -d "./frontend/src-tauri/runtime/jre" ]; then
echo "❌ Failed to create JLink runtime"
exit 1
fi
# Test the bundled runtime
if [ -f "./frontend/src-tauri/runtime/jre/bin/java" ]; then
RUNTIME_VERSION=$(./frontend/src-tauri/runtime/jre/bin/java --version 2>&1 | head -n 1)
echo "✅ Custom JRE created successfully: $RUNTIME_VERSION"
else
echo "❌ Custom JRE executable not found"
exit 1
fi
# Calculate runtime size
RUNTIME_SIZE=$(du -sh ./frontend/src-tauri/runtime/jre | cut -f1)
echo "📊 Custom JRE size: $RUNTIME_SIZE"
- name: Prepare desktop build
run: task desktop:prepare
env:
MAVEN_USER: ${{ secrets.MAVEN_USER }}
MAVEN_PASSWORD: ${{ secrets.MAVEN_PASSWORD }}
MAVEN_PUBLIC_URL: ${{ secrets.MAVEN_PUBLIC_URL }}
DISABLE_ADDITIONAL_FEATURES: true
- name: Install frontend dependencies
working-directory: ./frontend
run: npm ci
# DigiCert KeyLocker Setup (Cloud HSM)
- name: Setup DigiCert KeyLocker
id: digicert-setup
if: ${{ matrix.platform == 'windows-latest' && env.SM_API_KEY != '' && (github.event_name == 'release' || github.ref == 'refs/heads/V2-master') }}
if: ${{ matrix.platform == 'windows-latest' && env.SM_API_KEY != '' && (github.event_name == 'release' || (github.event_name == 'workflow_dispatch' && github.event.inputs.sign != 'false') || github.ref == 'refs/heads/V2-master') }}
uses: digicert/ssm-code-signing@1d820463733701cf1484c7eb5d7d24a15ca2c454 # v1.2.1
env:
SM_API_KEY: ${{ secrets.SM_API_KEY }}
@@ -311,7 +254,7 @@ jobs:
SM_HOST: ${{ secrets.SM_HOST }}
- name: Setup DigiCert KeyLocker Certificate
if: ${{ matrix.platform == 'windows-latest' && env.SM_API_KEY != '' && (github.event_name == 'release' || github.ref == 'refs/heads/V2-master') }}
if: ${{ matrix.platform == 'windows-latest' && env.SM_API_KEY != '' && (github.event_name == 'release' || (github.event_name == 'workflow_dispatch' && github.event.inputs.sign != 'false') || github.ref == 'refs/heads/V2-master') }}
shell: pwsh
run: |
Write-Host "Setting up DigiCert KeyLocker environment..."
@@ -346,7 +289,7 @@ jobs:
# Traditional PFX Certificate Import (fallback if KeyLocker not configured)
- name: Import Windows Code Signing Certificate
if: ${{ matrix.platform == 'windows-latest' && env.SM_API_KEY == '' && (github.event_name == 'release' || github.ref == 'refs/heads/V2-master') }}
if: ${{ matrix.platform == 'windows-latest' && env.SM_API_KEY == '' && (github.event_name == 'release' || (github.event_name == 'workflow_dispatch' && github.event.inputs.sign != 'false') || github.ref == 'refs/heads/V2-master') }}
env:
WINDOWS_CERTIFICATE: ${{ secrets.WINDOWS_CERTIFICATE }}
WINDOWS_CERTIFICATE_PASSWORD: ${{ secrets.WINDOWS_CERTIFICATE_PASSWORD }}
@@ -377,7 +320,7 @@ jobs:
}
- name: Import Apple Developer Certificate
if: (matrix.platform == 'macos-15' || matrix.platform == 'macos-15-intel') && (github.event_name == 'release' || github.ref == 'refs/heads/V2-master')
if: (matrix.platform == 'macos-15' || matrix.platform == 'macos-15-intel') && (github.event_name == 'release' || (github.event_name == 'workflow_dispatch' && github.event.inputs.sign != 'false') || github.ref == 'refs/heads/V2-master')
env:
APPLE_CERTIFICATE: ${{ secrets.APPLE_CERTIFICATE }}
APPLE_CERTIFICATE_PASSWORD: ${{ secrets.APPLE_CERTIFICATE_PASSWORD }}
@@ -398,7 +341,7 @@ jobs:
rm certificate.p12
- name: Verify Certificate
if: (matrix.platform == 'macos-15' || matrix.platform == 'macos-15-intel') && (github.event_name == 'release' || github.ref == 'refs/heads/V2-master')
if: (matrix.platform == 'macos-15' || matrix.platform == 'macos-15-intel') && (github.event_name == 'release' || (github.event_name == 'workflow_dispatch' && github.event.inputs.sign != 'false') || github.ref == 'refs/heads/V2-master')
run: |
echo "Verifying Apple Developer Certificate..."
KEYCHAIN_PATH=$RUNNER_TEMP/app-signing.keychain-db
@@ -409,6 +352,82 @@ jobs:
echo "APPLE_SIGNING_IDENTITY=$CERT_ID" >> $GITHUB_ENV
echo "Certificate imported successfully."
# Pre-flight: verify smctl can talk to DigiCert and sync cert before we sign.
# Mirrors the setup from working public Tauri+KeyLocker repos (Labric, Meetily).
# Without this, signCommand failures are opaque (Tauri captures but drops
# smctl's stderr) - running these loudly surfaces auth/env/keypair issues.
- name: Preflight smctl
if: ${{ matrix.platform == 'windows-latest' && env.SM_API_KEY != '' && (github.event_name == 'release' || (github.event_name == 'workflow_dispatch' && github.event.inputs.sign != 'false') || github.ref == 'refs/heads/V2-master') }}
shell: pwsh
env:
KEYPAIR_ALIAS: ${{ secrets.SM_KEYPAIR_ALIAS }}
run: |
& smctl healthcheck
if ($LASTEXITCODE -ne 0) { Write-Host "[ERROR] smctl healthcheck failed"; exit 1 }
& smctl keypair ls
if ($LASTEXITCODE -ne 0) { Write-Host "[ERROR] smctl keypair ls failed"; exit 1 }
& smctl windows certsync --keypair-alias "$env:KEYPAIR_ALIAS"
if ($LASTEXITCODE -ne 0) { Write-Host "[WARN] smctl windows certsync returned non-zero - continuing" }
Write-Host "[SUCCESS] smctl preflight passed"
# Write platform-specific Tauri config that adds signCommand for Windows.
# Tauri auto-merges tauri.windows.conf.json with tauri.conf.json (RFC 7396).
# Tauri calls this command on every binary BEFORE bundling into the MSI,
# substituting %1 with the file path.
#
# Why OBJECT form (cmd + args) instead of string:
# Tauri's string-form parser does a naive split(' ') with no shell/quote handling.
# Args with spaces or quote characters get mangled. The object form passes each
# arg directly to Rust's Command::arg which handles Windows CreateProcess quoting.
#
# Why --keypair-alias instead of --fingerprint:
# --fingerprint requires smctl windows certsync to have synced the cert to the
# Windows cert store first. --keypair-alias goes direct through PKCS11 and works
# without certsync. All real-world working Tauri+smctl examples use this flag.
#
# smctl reads SM_HOST, SM_API_KEY, SM_CLIENT_CERT_FILE, SM_CLIENT_CERT_PASSWORD
# from env (set by prior DigiCert setup step). No --config-file needed.
- name: Configure Windows code signing
if: ${{ matrix.platform == 'windows-latest' && env.SM_API_KEY != '' && (github.event_name == 'release' || (github.event_name == 'workflow_dispatch' && github.event.inputs.sign != 'false') || github.ref == 'refs/heads/V2-master') }}
shell: bash
env:
KEYPAIR_ALIAS: ${{ secrets.SM_KEYPAIR_ALIAS }}
run: |
cat > ./frontend/src-tauri/tauri.windows.conf.json <<EOF
{
"bundle": {
"windows": {
"signCommand": {
"cmd": "smctl",
"args": ["sign", "--keypair-alias", "${KEYPAIR_ALIAS}", "--input", "%1", "--verbose"]
}
}
}
}
EOF
echo "Generated tauri.windows.conf.json (alias masked):"
sed "s/${KEYPAIR_ALIAS}/***/g" ./frontend/src-tauri/tauri.windows.conf.json
- name: Import release GPG signing key (Linux)
if: matrix.platform == 'ubuntu-22.04'
env:
RELEASE_GPG_PRIVATE_KEY: ${{ secrets.RELEASE_GPG_PRIVATE_KEY }}
run: |
echo "$RELEASE_GPG_PRIVATE_KEY" | gpg --batch --import
gpg --list-secret-keys --keyid-format=long
- name: Make libjvm discoverable for linuxdeploy (Linux AppImage)
if: matrix.platform == 'ubuntu-22.04'
run: |
JAVA_LIBJVM="$JAVA_HOME/lib/server/libjvm.so"
if [ -f "$JAVA_LIBJVM" ]; then
sudo ln -sf "$JAVA_LIBJVM" /usr/lib/libjvm.so
echo "Linked libjvm from $JAVA_LIBJVM -> /usr/lib/libjvm.so"
else
echo "libjvm not found at $JAVA_LIBJVM"
exit 1
fi
- name: Build Tauri app
uses: tauri-apps/tauri-action@51a9f1156b33df106d827c3a78f8f894946c5faa # v0.5.25
env:
@@ -419,114 +438,115 @@ jobs:
APPLE_ID: ${{ secrets.APPLE_ID }}
APPLE_PASSWORD: ${{ secrets.APPLE_ID_PASSWORD }}
APPLE_TEAM_ID: ${{ secrets.APPLE_TEAM_ID }}
APPIMAGETOOL_SIGN_PASSPHRASE: ${{ secrets.APPIMAGETOOL_SIGN_PASSPHRASE }}
# AppImage signing — three env vars work together:
# SIGN=1 tells linuxdeploy-plugin-appimage to forward --sign to appimagetool
# APPIMAGETOOL_SIGN_PASSPHRASE appimagetool uses this to unlock the GPG key non-interactively
# SIGN_KEY appimagetool picks the key matching this fingerprint
# Without SIGN=1, the other two are ignored and the AppImage is built unsigned even if a key is present.
SIGN: "1"
APPIMAGETOOL_SIGN_PASSPHRASE: ${{ secrets.RELEASE_GPG_PASSPHRASE }}
SIGN_KEY: ${{ vars.RELEASE_GPG_FINGERPRINT }}
TAURI_SIGNING_PRIVATE_KEY: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY }}
TAURI_SIGNING_PRIVATE_KEY_PASSWORD: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY_PASSWORD }}
VITE_SUPABASE_PUBLISHABLE_DEFAULT_KEY: ${{ secrets.VITE_SUPABASE_PUBLISHABLE_DEFAULT_KEY || 'sb_publishable_UHz2SVRF5mvdrPHWkRteyA_yNlZTkYb' }}
VITE_SAAS_SERVER_URL: ${{ secrets.VITE_SAAS_SERVER_URL || 'https://app.stirlingpdf.com' }}
VITE_SAAS_BACKEND_API_URL: ${{ secrets.VITE_SAAS_BACKEND_API_URL || 'https://api.stirlingpdf.com' }}
# Only enable Windows signing in Tauri when on release or V2-master
SIGN: ${{ (github.event_name == 'release' || github.ref == 'refs/heads/V2-master') && (env.SM_API_KEY == '' && env.WINDOWS_CERTIFICATE != '') && '1' || '0' }}
# DigiCert KeyLocker env vars consumed by smctl during signCommand
SM_CODE_SIGNING_CERT_SHA1_HASH: ${{ secrets.SM_CODE_SIGNING_CERT_SHA1_HASH }}
CI: true
with:
projectPath: ./frontend
tauriScript: npx tauri
args: ${{ matrix.args }}
# Sign with DigiCert KeyLocker (post-build)
- name: Sign Windows binaries with DigiCert KeyLocker
if: ${{ matrix.platform == 'windows-latest' && env.SM_API_KEY != '' && (github.event_name == 'release' || github.ref == 'refs/heads/V2-master') }}
- name: Clear release GPG key from runner keyring (Linux)
if: always() && matrix.platform == 'ubuntu-22.04'
env:
RELEASE_GPG_FINGERPRINT: ${{ vars.RELEASE_GPG_FINGERPRINT }}
run: |
if [ -n "$RELEASE_GPG_FINGERPRINT" ]; then
gpg --batch --yes --delete-secret-keys "$RELEASE_GPG_FINGERPRINT" || true
gpg --batch --yes --delete-keys "$RELEASE_GPG_FINGERPRINT" || true
fi
# Verify the MSI (outer wrapper users download) AND the inner exe extracted
# from it (what actually gets installed and what AV scans). We don't check
# target/.../release/stirling-pdf.exe - that's Tauri's intermediate build
# artifact. Tauri signs a COPY when bundling into the MSI and leaves the raw
# cargo output unsigned, so checking it produces false negatives.
- name: Verify Windows Code Signature
if: ${{ matrix.platform == 'windows-latest' && env.SM_API_KEY != '' && (github.event_name == 'release' || (github.event_name == 'workflow_dispatch' && github.event.inputs.sign != 'false') || github.ref == 'refs/heads/V2-master') }}
shell: pwsh
run: |
Write-Host "=== DigiCert KeyLocker Signing ==="
$allSigned = $true
# Test smctl connectivity first
Write-Host "Testing smctl connection..."
$healthCheck = & smctl healthcheck 2>&1
if ($LASTEXITCODE -eq 0) {
Write-Host "[SUCCESS] Connected to DigiCert KeyLocker"
} else {
Write-Host "[ERROR] Failed to connect to DigiCert KeyLocker"
Write-Host $healthCheck
exit 1
}
Write-Host ""
# Sync certificates to Windows certificate store
Write-Host "Syncing certificates to Windows certificate store..."
$syncOutput = & smctl windows certsync 2>&1
Write-Host "Cert sync result: $syncOutput"
Write-Host ""
# Find only the files we need to sign
$filesToSign = @()
# Main application executable
$mainExe = Get-ChildItem -Path "./frontend/src-tauri/target/x86_64-pc-windows-msvc/release" -Filter "stirling-pdf.exe" -File -ErrorAction SilentlyContinue
if ($mainExe) { $filesToSign += $mainExe }
# MSI installer
# Check MSI installer (outer wrapper - what users download)
$msiFiles = Get-ChildItem -Path "./frontend/src-tauri/target" -Filter "*.msi" -Recurse -File
$filesToSign += $msiFiles
if ($filesToSign.Count -eq 0) {
Write-Host "[ERROR] No files found to sign"
if ($msiFiles.Count -eq 0) {
Write-Host "[ERROR] No MSI found under target/"
exit 1
}
Write-Host "Found $($filesToSign.Count) files to sign:"
foreach ($f in $filesToSign) { Write-Host " - $($f.Name)" }
Write-Host ""
$signedCount = 0
foreach ($file in $filesToSign) {
Write-Host "Signing: $($file.Name)"
# Get PKCS11 config file path
$pkcs11Config = $env:PKCS11_CONFIG
if (-not $pkcs11Config) {
Write-Host "[ERROR] PKCS11_CONFIG environment variable not set"
exit 1
foreach ($msi in $msiFiles) {
$sig = Get-AuthenticodeSignature -FilePath $msi.FullName
Write-Host "MSI: Status=$($sig.Status), Signer=$($sig.SignerCertificate.Subject)"
if ($sig.Status -ne "Valid") {
Write-Host "[ERROR] MSI is not signed"
$allSigned = $false
}
Write-Host "Using PKCS11 config: $pkcs11Config"
# Try signing with certificate fingerprint first (if available)
$fingerprint = "${{ secrets.SM_CODE_SIGNING_CERT_SHA1_HASH }}"
if ($fingerprint -and $fingerprint -ne "") {
Write-Host "Attempting to sign with certificate fingerprint..."
$output = & smctl sign --fingerprint "$fingerprint" --input "$($file.FullName)" --config-file "$pkcs11Config" --verbose 2>&1
$exitCode = $LASTEXITCODE
} else {
Write-Host "No fingerprint provided, using keypair alias..."
$output = & smctl sign --keypair-alias "${{ secrets.SM_KEYPAIR_ALIAS }}" --input "$($file.FullName)" --config-file "$pkcs11Config" --verbose 2>&1
$exitCode = $LASTEXITCODE
}
Write-Host "Exit code: $exitCode"
Write-Host "Output: $output"
if ($output -match "FAILED" -or $output -match "error" -or $output -match "Error") {
Write-Host "[ERROR] Signing failed for $($file.Name)"
exit 1
}
if ($exitCode -ne 0) {
Write-Host "[ERROR] Failed to sign $($file.Name)"
Write-Host "Full error output:"
Write-Host $output
exit 1
}
$signedCount++
Write-Host "[SUCCESS] Signed: $($file.Name)"
Write-Host ""
}
Write-Host "=== Summary ==="
Write-Host "[SUCCESS] Signed $signedCount/$($filesToSign.Count) files successfully"
# Extract MSI and verify the inner exe (the file that actually gets installed).
# This is the critical check - AV flags the installed exe at runtime.
$msi = $msiFiles[0].FullName
$extractDir = Join-Path $env:RUNNER_TEMP "msi-verify"
if (Test-Path $extractDir) { Remove-Item $extractDir -Recurse -Force }
$proc = Start-Process msiexec.exe -ArgumentList '/a', $msi, '/qn', "TARGETDIR=$extractDir" -Wait -PassThru -NoNewWindow
if ($proc.ExitCode -eq 0) {
$innerExe = Get-ChildItem -Path $extractDir -Filter "stirling-pdf.exe" -Recurse -File | Select-Object -First 1
if ($innerExe) {
$sig = Get-AuthenticodeSignature -FilePath $innerExe.FullName
Write-Host "Inner EXE (from MSI): Status=$($sig.Status), Signer=$($sig.SignerCertificate.Subject)"
if ($sig.Status -ne "Valid") {
Write-Host "[ERROR] Inner exe extracted from MSI is NOT signed - AV will flag this at runtime"
$allSigned = $false
}
} else {
Write-Host "[ERROR] Could not find stirling-pdf.exe inside MSI"
$allSigned = $false
}
} else {
Write-Host "[ERROR] Failed to extract MSI for verification (exit code: $($proc.ExitCode))"
$allSigned = $false
}
if (-not $allSigned) {
Write-Host "[ERROR] Signature verification failed"
exit 1
}
Write-Host "[SUCCESS] MSI and installed exe are properly signed"
# Dump smctl log files on failure. Tauri's signCommand captures smctl output
# but drops stderr when the command exits non-zero, making failures opaque.
# The real errors live in smctl's log files - surface them here for debugging.
- name: Dump smctl logs on failure
if: ${{ failure() && matrix.platform == 'windows-latest' && env.SM_API_KEY != '' }}
shell: pwsh
run: |
$logDir = "$env:USERPROFILE\.signingmanager\logs"
if (Test-Path $logDir) {
Get-ChildItem $logDir | ForEach-Object {
Write-Host "=== $($_.FullName) ==="
Get-Content $_.FullName -Tail 200
Write-Host ""
}
} else {
Write-Host "smctl log directory not found at $logDir"
}
# Rename + Upload: use always() so artifacts are still collected when verify
# fails - we need them to manually inspect what actually came out of the build.
- name: Rename artifacts
if: always() && steps.digicert-setup.conclusion != 'failure'
shell: bash
run: |
mkdir -p ./dist
@@ -534,17 +554,20 @@ jobs:
# Find and rename artifacts based on platform
if [ "${{ matrix.platform }}" = "windows-latest" ]; then
find . -name "*.exe" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.exe" \;
# Only ship the MSI installer on Windows. The loose exe and WiX toolset exes
# are not the user-facing installer - the MSI contains the signed inner exe.
find . -name "*.msi" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.msi" \;
elif [ "${{ matrix.platform }}" = "macos-15" ] || [ "${{ matrix.platform }}" = "macos-15-intel" ]; then
find . -name "*.dmg" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.dmg" \;
find . -name "*.app" -exec cp -r {} "../../../dist/Stirling-PDF-${{ matrix.name }}.app" \;
else
find . -name "*.deb" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.deb" \;
find . -name "*.rpm" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.rpm" \;
find . -name "*.AppImage" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.AppImage" \;
fi
- name: Upload build artifacts
if: always() && steps.digicert-setup.conclusion != 'failure'
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
with:
name: Stirling-PDF-${{ matrix.name }}
@@ -600,6 +623,7 @@ jobs:
./artifacts/**/*.msi
./artifacts/**/*.dmg
./artifacts/**/*.deb
./artifacts/**/*.rpm
./artifacts/**/*.AppImage
draft: false
prerelease: false
+4 -8
View File
@@ -32,17 +32,13 @@ jobs:
cache: "npm"
cache-dependency-path: frontend/package-lock.json
- name: Install frontend dependencies
run: cd frontend && npm ci
- name: Generate icons
run: cd frontend && node scripts/generate-icons.js
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
- name: Install all Playwright browsers
run: cd frontend && npx playwright install --with-deps
run: task frontend:test:e2e:install
- name: Run E2E tests (all browsers)
run: cd frontend && npx playwright test
run: task frontend:test:e2e
- name: Upload Playwright report
if: always()
+197
View File
@@ -0,0 +1,197 @@
name: Update Package Manager Manifests
on:
# release:
# types: [released]
workflow_dispatch:
inputs:
version:
description: "Version to test (e.g. 2.9.2 — no v prefix)"
required: true
type: string
dry_run:
description: "Skip the git push at the end (safe test)"
type: boolean
default: true
permissions:
contents: read
jobs:
get-release-info:
runs-on: ubuntu-latest
outputs:
version: ${{ steps.info.outputs.version }}
dmg_arm64_sha256: ${{ steps.hashes.outputs.dmg_arm64_sha256 }}
dmg_x86_64_sha256: ${{ steps.hashes.outputs.dmg_x86_64_sha256 }}
msi_sha256: ${{ steps.hashes.outputs.msi_sha256 }}
deb_sha256: ${{ steps.hashes.outputs.deb_sha256 }}
jar_sha256: ${{ steps.hashes.outputs.jar_sha256 }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@v2
with:
egress-policy: audit
- name: Extract version from tag or manual input
id: info
env:
DISPATCH_VERSION: ${{ inputs.version }}
RELEASE_TAG: ${{ github.event.release.tag_name }}
run: |
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
VERSION="$DISPATCH_VERSION"
else
VERSION="$RELEASE_TAG"
fi
VERSION="${VERSION#v}"
echo "version=$VERSION" >> "$GITHUB_OUTPUT"
- name: Download release assets and compute SHA256
id: hashes
env:
VERSION: ${{ steps.info.outputs.version }}
GH_TOKEN: ${{ github.token }}
run: |
BASE="https://github.com/Stirling-Tools/Stirling-PDF/releases/download/v${VERSION}"
download_sha256() {
local url="$1"
local file
file=$(basename "$url")
curl -fsSL --retry 3 -o "$file" "$url"
sha256sum "$file" | awk '{print $1}'
}
DMG_ARM64_SHA=$(download_sha256 "${BASE}/Stirling-PDF-macos-aarch64.dmg")
DMG_X64_SHA=$(download_sha256 "${BASE}/Stirling-PDF-macos-x86_64.dmg")
MSI_SHA=$(download_sha256 "${BASE}/Stirling-PDF-windows-x86_64.msi")
DEB_SHA=$(download_sha256 "${BASE}/Stirling-PDF-linux-x86_64.deb")
JAR_SHA=$(download_sha256 "${BASE}/Stirling-PDF-with-login.jar")
echo "dmg_arm64_sha256=$DMG_ARM64_SHA" >> "$GITHUB_OUTPUT"
echo "dmg_x86_64_sha256=$DMG_X64_SHA" >> "$GITHUB_OUTPUT"
echo "msi_sha256=$MSI_SHA" >> "$GITHUB_OUTPUT"
echo "deb_sha256=$DEB_SHA" >> "$GITHUB_OUTPUT"
echo "jar_sha256=$JAR_SHA" >> "$GITHUB_OUTPUT"
update-homebrew:
needs: get-release-info
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@v2
with:
egress-policy: audit
- name: Checkout homebrew tap
uses: actions/checkout@v4
with:
repository: Stirling-Tools/homebrew-stirling-pdf
token: ${{ secrets.HOMEBREW_TAP_TOKEN }}
path: homebrew-tap
- name: Update cask (stirling-pdf.rb)
env:
VERSION: ${{ needs.get-release-info.outputs.version }}
ARM64_SHA: ${{ needs.get-release-info.outputs.dmg_arm64_sha256 }}
X64_SHA: ${{ needs.get-release-info.outputs.dmg_x86_64_sha256 }}
run: |
CASK="homebrew-tap/Casks/stirling-pdf.rb"
sed -i "s/version \".*\"/version \"${VERSION}\"/" "$CASK"
# Update ARM64 sha256 (line following on_arm block)
awk -v arm="$ARM64_SHA" -v x64="$X64_SHA" '
/on_arm/ { in_arm=1 }
/on_intel/ { in_arm=0; in_intel=1 }
/end/ { in_arm=0; in_intel=0 }
in_arm && /sha256/ { sub(/sha256 ".*"/, "sha256 \"" arm "\"") }
in_intel && /sha256/ { sub(/sha256 ".*"/, "sha256 \"" x64 "\"") }
{ print }
' "$CASK" > tmp && mv tmp "$CASK"
- name: Update formula (stirling-pdf-server.rb)
env:
VERSION: ${{ needs.get-release-info.outputs.version }}
JAR_SHA: ${{ needs.get-release-info.outputs.jar_sha256 }}
run: |
FORMULA="homebrew-tap/Formula/stirling-pdf-server.rb"
sed -i "s/version \".*\"/version \"${VERSION}\"/" "$FORMULA"
sed -i "s/sha256 \".*\"/sha256 \"${JAR_SHA}\"/" "$FORMULA"
- name: Show homebrew tap diff (for dry-run visibility)
working-directory: homebrew-tap
run: |
echo "--- diff --stat ---"
git diff --stat
echo "--- full diff ---"
git diff
- name: Commit and push homebrew tap updates
if: ${{ github.event_name == 'release' || inputs.dry_run == false }}
working-directory: homebrew-tap
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git add Casks/stirling-pdf.rb Formula/stirling-pdf-server.rb
git diff --cached --quiet && echo "No changes" && exit 0
git commit -m "chore: bump Stirling-PDF to v${{ needs.get-release-info.outputs.version }}"
git push
update-scoop:
needs: get-release-info
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@v2
with:
egress-policy: audit
- name: Checkout Scoop bucket (shared with Homebrew tap)
uses: actions/checkout@v4
with:
repository: Stirling-Tools/homebrew-stirling-pdf
token: ${{ secrets.SCOOP_BUCKET_TOKEN }}
path: scoop-bucket
- name: Update stirling-pdf.json
env:
VERSION: ${{ needs.get-release-info.outputs.version }}
MSI_SHA: ${{ needs.get-release-info.outputs.msi_sha256 }}
run: |
MANIFEST="scoop-bucket/scoop/stirling-pdf.json"
jq --arg v "$VERSION" --arg h "$MSI_SHA" \
'.version = $v | .architecture["64bit"].url = "https://github.com/Stirling-Tools/Stirling-PDF/releases/download/v\($v)/Stirling-PDF-windows-x86_64.msi" | .architecture["64bit"].hash = $h' \
"$MANIFEST" > tmp.json && mv tmp.json "$MANIFEST"
- name: Update stirling-pdf-server.json
env:
VERSION: ${{ needs.get-release-info.outputs.version }}
JAR_SHA: ${{ needs.get-release-info.outputs.jar_sha256 }}
run: |
MANIFEST="scoop-bucket/scoop/stirling-pdf-server.json"
jq --arg v "$VERSION" --arg h "$JAR_SHA" \
'.version = $v | .url = "https://github.com/Stirling-Tools/Stirling-PDF/releases/download/v\($v)/Stirling-PDF-with-login.jar" | .hash = $h' \
"$MANIFEST" > tmp.json && mv tmp.json "$MANIFEST"
- name: Show Scoop bucket diff (for dry-run visibility)
working-directory: scoop-bucket
run: |
echo "--- diff --stat ---"
git diff --stat
echo "--- full diff ---"
git diff
- name: Commit and push Scoop bucket updates
if: ${{ github.event_name == 'release' || inputs.dry_run == false }}
working-directory: scoop-bucket
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git add scoop/stirling-pdf.json scoop/stirling-pdf-server.json
git diff --cached --quiet && echo "No changes" && exit 0
git commit -m "chore: bump Stirling-PDF to v${{ needs.get-release-info.outputs.version }}"
git push
+2 -55
View File
@@ -2,7 +2,7 @@ name: Pre-commit
on:
workflow_dispatch:
push:
pull_request:
branches:
- main
@@ -16,9 +16,6 @@ jobs:
# Prevents sdist builds → no tar extraction
PIP_ONLY_BINARY: ":all:"
PIP_DISABLE_PIP_VERSION_CHECK: "1"
permissions:
contents: write
pull-requests: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@58077d3c7e43986b6b15fba718e8ea69e387dfcc # v2.15.1
@@ -31,13 +28,6 @@ jobs:
fetch-depth: 0
persist-credentials: false
- name: Setup GitHub App Bot
id: setup-bot
uses: ./.github/actions/setup-bot
with:
app-id: ${{ secrets.GH_APP_ID }}
private-key: ${{ secrets.GH_APP_PRIVATE_KEY }}
- name: Set up Python
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
@@ -57,47 +47,4 @@ jobs:
pre-commit run gitleaks --all-files -c .pre-commit-config.yaml
pre-commit run end-of-file-fixer --all-files -c .pre-commit-config.yaml
pre-commit run trailing-whitespace --all-files -c .pre-commit-config.yaml
continue-on-error: true
- name: Set up JDK 25
uses: actions/setup-java@be666c2fcd27ec809703dec50e508c2fdc7f6654 # v5.2.0
with:
java-version: "25"
distribution: "temurin"
- name: Setup Gradle
uses: gradle/actions/setup-gradle@f29f5a9d7b09a7c6b29859002d29d24e1674c884 # v5.0.1
with:
gradle-version: 9.3.1
- name: Build with Gradle
run: ./gradlew build
env:
MAVEN_USER: ${{ secrets.MAVEN_USER }}
MAVEN_PASSWORD: ${{ secrets.MAVEN_PASSWORD }}
MAVEN_PUBLIC_URL: ${{ secrets.MAVEN_PUBLIC_URL }}
- name: git add
run: |
git add .
git diff --staged --quiet || echo "CHANGES_DETECTED=true" >> $GITHUB_ENV
- name: Create Pull Request
if: env.CHANGES_DETECTED == 'true'
uses: peter-evans/create-pull-request@c0f553fe549906ede9cf27b5156039d195d2ece0 # v8.1.0
with:
token: ${{ steps.setup-bot.outputs.token }}
commit-message: ":file_folder: pre-commit"
committer: ${{ steps.setup-bot.outputs.committer }}
author: ${{ steps.setup-bot.outputs.committer }}
signoff: true
branch: pre-commit
title: "🤖 format everything with pre-commit by ${{ steps.setup-bot.outputs.app-slug }}"
body: |
Auto-generated by [create-pull-request][1] with **${{ steps.setup-bot.outputs.app-slug }}**
[1]: https://github.com/peter-evans/create-pull-request
draft: false
delete-branch: true
labels: github-actions
sign-commits: true
git diff --exit-code
+3
View File
@@ -4,6 +4,7 @@ on:
push:
branches:
- baseDockerImage
- accessIssueFix
workflow_dispatch:
inputs:
version:
@@ -34,6 +35,8 @@ jobs:
run: |
if [ "${{ github.event_name }}" == "workflow_dispatch" ]; then
VERSION="${{ github.event.inputs.version }}"
elif [ "${{ github.ref_name }}" == "accessIssueFix" ]; then
VERSION="1.0.3"
else
VERSION="1.0.0"
fi
+2
View File
@@ -64,6 +64,8 @@ jobs:
id: buildx
uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3.12.0
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
- name: Get version number
id: versionNumber
run: echo "versionNumber=$(./gradlew printVersion --quiet | tail -1)" >> $GITHUB_OUTPUT
+93
View File
@@ -0,0 +1,93 @@
name: Rollback Latest Tags to Version
on:
workflow_dispatch:
inputs:
version:
description: "Version to rollback to (e.g. 2.8.0)"
required: true
type: string
permissions:
contents: read
jobs:
rollback:
runs-on: ubuntu-latest
permissions:
packages: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@58077d3c7e43986b6b15fba718e8ea69e387dfcc # v2.15.1
with:
egress-policy: audit
- name: Install crane
uses: imjasonh/setup-crane@31b88afe9de28ae0ffa220711af4b60be9435f6e # v0.4
- name: Login to Docker Hub
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3.7.0
with:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_API }}
- name: Login to GitHub Container Registry
uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3.7.0
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ github.token }}
- name: Convert repository owner to lowercase
id: repoowner
run: echo "lowercase=$(echo ${{ github.repository_owner }} | awk '{print tolower($0)}')" >> $GITHUB_OUTPUT
- name: Rollback all latest tags to v${{ inputs.version }}
env:
VERSION: ${{ inputs.version }}
DOCKER_HUB_USERNAME: ${{ secrets.DOCKER_HUB_USERNAME }}
DOCKER_HUB_ORG_USERNAME: ${{ secrets.DOCKER_HUB_ORG_USERNAME }}
REPO_OWNER: ${{ steps.repoowner.outputs.lowercase }}
run: |
set -euo pipefail
IMAGES=(
"${DOCKER_HUB_USERNAME}/s-pdf"
"ghcr.io/${REPO_OWNER}/s-pdf"
"ghcr.io/${REPO_OWNER}/stirling-pdf"
"${DOCKER_HUB_ORG_USERNAME}/stirling-pdf"
)
VARIANTS=(
"${VERSION}:latest"
"${VERSION}-fat:latest-fat"
"${VERSION}-ultra-lite:latest-ultra-lite"
)
FAILED=0
for image in "${IMAGES[@]}"; do
for variant in "${VARIANTS[@]}"; do
SOURCE_TAG="${variant%%:*}"
TARGET_TAG="${variant##*:}"
echo "::group::${image} — ${SOURCE_TAG} → ${TARGET_TAG}"
if crane manifest "${image}:${SOURCE_TAG}" > /dev/null 2>&1; then
crane cp "${image}:${SOURCE_TAG}" "${image}:${TARGET_TAG}"
echo "✅ ${image}:${TARGET_TAG} now points to ${SOURCE_TAG}"
else
echo "::warning::⚠️ ${image}:${SOURCE_TAG} not found, skipping"
FAILED=1
fi
echo "::endgroup::"
done
done
if [ "$FAILED" -ne 0 ]; then
echo "::warning::Some source tags were not found. This is expected if not all variants exist for this version."
fi
echo ""
echo "🎉 Rollback to ${VERSION} complete!"
+2
View File
@@ -56,6 +56,8 @@ jobs:
SWAGGERHUB_API_KEY: ${{ secrets.SWAGGERHUB_API_KEY }}
SWAGGERHUB_USER: "Frooodle"
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
- name: Get version number
id: versionNumber
run: echo "versionNumber=$(./gradlew printVersion --quiet | tail -1)" >> $GITHUB_OUTPUT
+128 -282
View File
@@ -128,86 +128,16 @@ jobs:
with:
gradle-version: 9.3.1
- name: Build Java backend with JLink
working-directory: ./
shell: bash
run: |
chmod +x ./gradlew
echo "🔧 Building Stirling-PDF JAR..."
# STIRLING_PDF_DESKTOP_UI=false ./gradlew bootJar --no-daemon
./gradlew build -x spotlessApply -x spotlessCheck -x test -x sonarqube
# Find the built JAR
STIRLING_JAR=$(ls app/core/build/libs/stirling-pdf-*.jar | head -n 1)
echo "✅ Built JAR: $STIRLING_JAR"
# Create Tauri directories
mkdir -p ./frontend/src-tauri/libs
mkdir -p ./frontend/src-tauri/runtime
# Copy JAR to Tauri libs
cp "$STIRLING_JAR" ./frontend/src-tauri/libs/
echo "✅ JAR copied to Tauri libs"
# Analyze JAR dependencies for jlink modules
echo "🔍 Analyzing JAR dependencies..."
if command -v jdeps &> /dev/null; then
DETECTED_MODULES=$(jdeps --print-module-deps --ignore-missing-deps "$STIRLING_JAR" 2>/dev/null || echo "")
if [ -n "$DETECTED_MODULES" ]; then
echo "📋 jdeps detected modules: $DETECTED_MODULES"
MODULES="$DETECTED_MODULES,java.compiler,java.instrument,java.management,java.naming,java.net.http,java.prefs,java.rmi,java.scripting,java.security.jgss,java.security.sasl,java.sql,java.transaction.xa,java.xml.crypto,jdk.crypto.ec,jdk.crypto.cryptoki,jdk.unsupported"
else
echo "⚠️ jdeps analysis failed, using predefined modules"
MODULES="java.base,java.compiler,java.desktop,java.instrument,java.logging,java.management,java.naming,java.net.http,java.prefs,java.rmi,java.scripting,java.security.jgss,java.security.sasl,java.sql,java.transaction.xa,java.xml,java.xml.crypto,jdk.crypto.ec,jdk.crypto.cryptoki,jdk.unsupported"
fi
else
echo "⚠️ jdeps not available, using predefined modules"
MODULES="java.base,java.compiler,java.desktop,java.instrument,java.logging,java.management,java.naming,java.net.http,java.prefs,java.rmi,java.scripting,java.security.jgss,java.security.sasl,java.sql,java.transaction.xa,java.xml,java.xml.crypto,jdk.crypto.ec,jdk.crypto.cryptoki,jdk.unsupported"
fi
# Create custom JRE with jlink (always rebuild)
echo "🔧 Creating custom JRE with jlink..."
echo "📋 Using modules: $MODULES"
# Remove any existing JRE
rm -rf ./frontend/src-tauri/runtime/jre
# Create the custom JRE
jlink \
--add-modules "$MODULES" \
--strip-debug \
--compress=2 \
--no-header-files \
--no-man-pages \
--output ./frontend/src-tauri/runtime/jre
if [ ! -d "./frontend/src-tauri/runtime/jre" ]; then
echo "❌ Failed to create JLink runtime"
exit 1
fi
# Test the bundled runtime
if [ -f "./frontend/src-tauri/runtime/jre/bin/java" ]; then
RUNTIME_VERSION=$(./frontend/src-tauri/runtime/jre/bin/java --version 2>&1 | head -n 1)
echo "✅ Custom JRE created successfully: $RUNTIME_VERSION"
else
echo "❌ Custom JRE executable not found"
exit 1
fi
# Calculate runtime size
RUNTIME_SIZE=$(du -sh ./frontend/src-tauri/runtime/jre | cut -f1)
echo "📊 Custom JRE size: $RUNTIME_SIZE"
- name: Setup Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
- name: Prepare desktop build
run: task desktop:prepare
env:
MAVEN_USER: ${{ secrets.MAVEN_USER }}
MAVEN_PASSWORD: ${{ secrets.MAVEN_PASSWORD }}
MAVEN_PUBLIC_URL: ${{ secrets.MAVEN_PUBLIC_URL }}
DISABLE_ADDITIONAL_FEATURES: true
- name: Install frontend dependencies
working-directory: ./frontend
run: npm ci
# DigiCert KeyLocker Setup (Cloud HSM)
- name: Setup DigiCert KeyLocker
id: digicert-setup
@@ -330,6 +260,58 @@ jobs:
echo "Available tools:"
ls -la /usr/bin/hd* || echo "No hd* tools found"
- name: Preflight smctl
if: ${{ matrix.platform == 'windows-latest' && env.SM_API_KEY != '' && github.ref == 'refs/heads/main' }}
shell: pwsh
env:
KEYPAIR_ALIAS: ${{ secrets.SM_KEYPAIR_ALIAS }}
run: |
& smctl healthcheck
if ($LASTEXITCODE -ne 0) { Write-Host "[ERROR] smctl healthcheck failed"; exit 1 }
& smctl keypair ls
if ($LASTEXITCODE -ne 0) { Write-Host "[ERROR] smctl keypair ls failed"; exit 1 }
& smctl windows certsync --keypair-alias "$env:KEYPAIR_ALIAS"
if ($LASTEXITCODE -ne 0) { Write-Host "[WARN] smctl windows certsync returned non-zero - continuing" }
- name: Configure Windows code signing
if: ${{ matrix.platform == 'windows-latest' && env.SM_API_KEY != '' && github.ref == 'refs/heads/main' }}
shell: bash
env:
KEYPAIR_ALIAS: ${{ secrets.SM_KEYPAIR_ALIAS }}
run: |
cat > ./frontend/src-tauri/tauri.windows.conf.json <<EOF
{
"bundle": {
"windows": {
"signCommand": {
"cmd": "smctl",
"args": ["sign", "--keypair-alias", "${KEYPAIR_ALIAS}", "--input", "%1", "--verbose"]
}
}
}
}
EOF
- name: Import release GPG signing key (Linux)
if: matrix.platform == 'ubuntu-22.04'
env:
RELEASE_GPG_PRIVATE_KEY: ${{ secrets.RELEASE_GPG_PRIVATE_KEY }}
run: |
echo "$RELEASE_GPG_PRIVATE_KEY" | gpg --batch --import
gpg --list-secret-keys --keyid-format=long
- name: Make libjvm discoverable for linuxdeploy (Linux AppImage)
if: matrix.platform == 'ubuntu-22.04'
run: |
JAVA_LIBJVM="$JAVA_HOME/lib/server/libjvm.so"
if [ -f "$JAVA_LIBJVM" ]; then
sudo ln -sf "$JAVA_LIBJVM" /usr/lib/libjvm.so
echo "Linked libjvm from $JAVA_LIBJVM -> /usr/lib/libjvm.so"
else
echo "libjvm not found at $JAVA_LIBJVM"
exit 1
fi
- name: Build Tauri app
uses: tauri-apps/tauri-action@51a9f1156b33df106d827c3a78f8f894946c5faa # v0.5.25
env:
@@ -340,178 +322,35 @@ jobs:
APPLE_ID: ${{ secrets.APPLE_ID }}
APPLE_PASSWORD: ${{ secrets.APPLE_ID_PASSWORD }}
APPLE_TEAM_ID: ${{ secrets.APPLE_TEAM_ID }}
APPIMAGETOOL_SIGN_PASSPHRASE: ${{ secrets.APPIMAGETOOL_SIGN_PASSPHRASE }}
# AppImage signing — three env vars work together:
# SIGN=1 tells linuxdeploy-plugin-appimage to forward --sign to appimagetool
# APPIMAGETOOL_SIGN_PASSPHRASE appimagetool uses this to unlock the GPG key non-interactively
# SIGN_KEY appimagetool picks the key matching this fingerprint
# Without SIGN=1, the other two are ignored and the AppImage is built unsigned even if a key is present.
SIGN: "1"
APPIMAGETOOL_SIGN_PASSPHRASE: ${{ secrets.RELEASE_GPG_PASSPHRASE }}
SIGN_KEY: ${{ vars.RELEASE_GPG_FINGERPRINT }}
TAURI_SIGNING_PRIVATE_KEY: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY }}
TAURI_SIGNING_PRIVATE_KEY_PASSWORD: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY_PASSWORD }}
VITE_SUPABASE_PUBLISHABLE_DEFAULT_KEY: ${{ secrets.VITE_SUPABASE_PUBLISHABLE_DEFAULT_KEY || 'sb_publishable_UHz2SVRF5mvdrPHWkRteyA_yNlZTkYb' }}
VITE_SAAS_SERVER_URL: ${{ secrets.VITE_SAAS_SERVER_URL || 'https://app.stirlingpdf.com' }}
VITE_SAAS_BACKEND_API_URL: ${{ secrets.VITE_SAAS_BACKEND_API_URL || 'https://api.stirlingpdf.com' }}
# Only enable Windows signing in Tauri when on main
SIGN: ${{ github.ref == 'refs/heads/main' && (env.SM_API_KEY == '' && env.WINDOWS_CERTIFICATE != '') && '1' || '0' }}
SM_CODE_SIGNING_CERT_SHA1_HASH: ${{ secrets.SM_CODE_SIGNING_CERT_SHA1_HASH }}
CI: true
with:
projectPath: ./frontend
tauriScript: npx tauri
args: ${{ matrix.args }}
# Sign with DigiCert KeyLocker (post-build)
- name: Sign Windows binaries with DigiCert KeyLocker
if: ${{ matrix.platform == 'windows-latest' && env.SM_API_KEY != '' && github.ref == 'refs/heads/main' }}
shell: pwsh
- name: Clear release GPG key from runner keyring (Linux)
if: always() && matrix.platform == 'ubuntu-22.04'
env:
RELEASE_GPG_FINGERPRINT: ${{ vars.RELEASE_GPG_FINGERPRINT }}
run: |
Write-Host "=== DigiCert KeyLocker Signing ==="
# Test smctl connectivity first
Write-Host "Testing smctl connection..."
$healthCheck = & smctl healthcheck 2>&1
if ($LASTEXITCODE -eq 0) {
Write-Host "[SUCCESS] Connected to DigiCert KeyLocker"
} else {
Write-Host "[ERROR] Failed to connect to DigiCert KeyLocker"
Write-Host $healthCheck
exit 1
}
Write-Host ""
# Sync certificates to Windows certificate store
Write-Host "Syncing certificates to Windows certificate store..."
$syncOutput = & smctl windows certsync 2>&1
Write-Host "Cert sync result: $syncOutput"
Write-Host ""
# List available certificates and check if they have certificates attached
Write-Host "Checking for available certificates..."
$certList = & smctl keypair ls 2>&1
Write-Host "Keypair list output:"
Write-Host $certList
Write-Host ""
# Parse the output to check certificate status
$lines = $certList -split "`n"
$foundKeypair = $false
$hasCertificate = $false
foreach ($line in $lines) {
if ($line -match "${{ secrets.SM_KEYPAIR_ALIAS }}") {
$foundKeypair = $true
Write-Host "[SUCCESS] Found keypair in list"
# Check if this line has certificate info (not just empty spaces after alias)
$parts = $line -split "\s+"
if ($parts.Count -gt 2 -and $parts[1] -ne "" -and $parts[1] -ne "CERTIFICATE") {
$hasCertificate = $true
Write-Host "[SUCCESS] Certificate is associated with keypair"
}
}
}
if (-not $foundKeypair) {
Write-Host "[ERROR] Keypair not found: ${{ secrets.SM_KEYPAIR_ALIAS }}"
Write-Host "Available keypairs are listed above"
Write-Host ""
Write-Host "Please verify:"
Write-Host " 1. Keypair alias is correct in GitHub secret"
Write-Host " 2. API key has access to this keypair"
exit 1
}
if (-not $hasCertificate) {
Write-Host "[ERROR] No certificate associated with keypair"
Write-Host "This usually means:"
Write-Host " 1. Certificate not yet synced to KeyLocker (run sync manually)"
Write-Host " 2. Certificate is pending approval"
Write-Host " 3. Certificate needs to be attached to the keypair"
Write-Host ""
Write-Host "Try running in DigiCert ONE portal:"
Write-Host " smctl keypair sync"
exit 1
}
Write-Host "[SUCCESS] Certificate check passed"
Write-Host ""
# Find only the files we need to sign (not build scripts)
$filesToSign = @()
# Main application executable
$mainExe = Get-ChildItem -Path "./frontend/src-tauri/target/x86_64-pc-windows-msvc/release" -Filter "stirling-pdf.exe" -File -ErrorAction SilentlyContinue
if ($mainExe) { $filesToSign += $mainExe }
# MSI installer
$msiFiles = Get-ChildItem -Path "./frontend/src-tauri/target" -Filter "*.msi" -Recurse -File
$filesToSign += $msiFiles
if ($filesToSign.Count -eq 0) {
Write-Host "[ERROR] No files found to sign"
exit 1
}
Write-Host "Found $($filesToSign.Count) files to sign:"
foreach ($f in $filesToSign) { Write-Host " - $($f.Name)" }
Write-Host ""
$signedCount = 0
foreach ($file in $filesToSign) {
Write-Host "Signing: $($file.Name)"
# Get PKCS11 config file path (set by DigiCert action)
$pkcs11Config = $env:PKCS11_CONFIG
if (-not $pkcs11Config) {
Write-Host "[ERROR] PKCS11_CONFIG environment variable not set"
Write-Host "DigiCert KeyLocker action may not have run correctly"
exit 1
}
Write-Host "Using PKCS11 config: $pkcs11Config"
# Try signing with certificate fingerprint first (if available)
$fingerprint = "${{ secrets.SM_CODE_SIGNING_CERT_SHA1_HASH }}"
if ($fingerprint -and $fingerprint -ne "") {
Write-Host "Attempting to sign with certificate fingerprint..."
$output = & smctl sign --fingerprint "$fingerprint" --input "$($file.FullName)" --config-file "$pkcs11Config" --verbose 2>&1
$exitCode = $LASTEXITCODE
} else {
Write-Host "No fingerprint provided, using keypair alias..."
# Use smctl to sign with keypair alias
$output = & smctl sign --keypair-alias "${{ secrets.SM_KEYPAIR_ALIAS }}" --input "$($file.FullName)" --config-file "$pkcs11Config" --verbose 2>&1
$exitCode = $LASTEXITCODE
}
Write-Host "Exit code: $exitCode"
Write-Host "Output: $output"
# Check if output contains "FAILED" even with exit code 0
if ($output -match "FAILED" -or $output -match "error" -or $output -match "Error") {
Write-Host ""
Write-Host "[ERROR] Signing failed for $($file.Name)"
Write-Host "[ERROR] smctl returned success but output indicates failure"
Write-Host ""
Write-Host "Possible issues:"
Write-Host " 1. Certificate not fully synced to KeyLocker (wait a few minutes)"
Write-Host " 2. Incorrect keypair alias"
Write-Host " 3. API key lacks signing permissions"
Write-Host ""
Write-Host "Please verify in DigiCert ONE portal:"
Write-Host " - Certificate status is 'Issued' (not Pending)"
Write-Host " - Keypair status is 'Online'"
Write-Host " - 'Can sign' is set to 'Yes'"
exit 1
}
if ($exitCode -ne 0) {
Write-Host "[ERROR] Failed to sign $($file.Name)"
Write-Host "Full error output:"
Write-Host $output
exit 1
}
$signedCount++
Write-Host "[SUCCESS] Signed: $($file.Name)"
Write-Host ""
}
Write-Host "=== Summary ==="
Write-Host "[SUCCESS] Signed $signedCount/$($filesToSign.Count) files successfully"
if [ -n "$RELEASE_GPG_FINGERPRINT" ]; then
gpg --batch --yes --delete-secret-keys "$RELEASE_GPG_FINGERPRINT" || true
gpg --batch --yes --delete-keys "$RELEASE_GPG_FINGERPRINT" || true
fi
- name: Verify notarization (macOS only)
if: matrix.platform == 'macos-15' || matrix.platform == 'macos-15-intel'
@@ -536,73 +375,80 @@ jobs:
# Find and rename artifacts based on platform
if [ "${{ matrix.platform }}" = "windows-latest" ]; then
find . -name "*.exe" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.exe" \;
# Only ship the MSI installer. The loose exe and WiX toolset exes
# are not the user-facing installer - the MSI contains the signed inner exe.
find . -name "*.msi" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.msi" \;
elif [ "${{ matrix.platform }}" = "macos-15" ] || [ "${{ matrix.platform }}" = "macos-15-intel" ]; then
find . -name "*.dmg" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.dmg" \;
else
find . -name "*.deb" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.deb" \;
find . -name "*.rpm" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.rpm" \;
find . -name "*.AppImage" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.AppImage" \;
fi
# Verify the MSI AND the inner exe extracted from it are signed.
# The inner exe is what gets installed on users' machines and what AV scans.
- name: Verify Windows Code Signature
if: matrix.platform == 'windows-latest' && github.ref == 'refs/heads/main'
if: matrix.platform == 'windows-latest' && env.SM_API_KEY != '' && github.ref == 'refs/heads/main'
shell: pwsh
run: |
Write-Host "Verifying Windows code signatures..."
$exePath = "./dist/Stirling-PDF-${{ matrix.name }}.exe"
$allSigned = $true
$msiPath = "./dist/Stirling-PDF-${{ matrix.name }}.msi"
$allSigned = $true
$usingKeyLocker = "${{ env.SM_API_KEY }}" -ne ""
$usingPfx = "${{ env.WINDOWS_CERTIFICATE }}" -ne ""
# Check EXE signature
if (Test-Path $exePath) {
$exeSig = Get-AuthenticodeSignature -FilePath $exePath
Write-Host "EXE Signature Status: $($exeSig.Status)"
Write-Host "EXE Signer: $($exeSig.SignerCertificate.Subject)"
Write-Host "EXE Timestamp: $($exeSig.TimeStamperCertificate.NotAfter)"
if ($exeSig.Status -ne "Valid") {
Write-Host "[WARNING] EXE is not properly signed (Status: $($exeSig.Status))"
if ($usingKeyLocker -or $usingPfx) {
Write-Host "[ERROR] Certificate was provided but signing failed"
$allSigned = $false
} else {
Write-Host "[INFO] Building unsigned binary (no certificate provided)"
}
} else {
Write-Host "[SUCCESS] EXE is properly signed"
}
}
# Check MSI signature
# Check MSI (outer wrapper)
if (Test-Path $msiPath) {
$msiSig = Get-AuthenticodeSignature -FilePath $msiPath
Write-Host "MSI Signature Status: $($msiSig.Status)"
Write-Host "MSI Signer: $($msiSig.SignerCertificate.Subject)"
Write-Host "MSI Timestamp: $($msiSig.TimeStamperCertificate.NotAfter)"
$sig = Get-AuthenticodeSignature -FilePath $msiPath
Write-Host "MSI: Status=$($sig.Status), Signer=$($sig.SignerCertificate.Subject)"
if ($sig.Status -ne "Valid") {
Write-Host "[ERROR] MSI is not signed"
$allSigned = $false
}
if ($msiSig.Status -ne "Valid") {
Write-Host "[WARNING] MSI is not properly signed (Status: $($msiSig.Status))"
if ($usingKeyLocker -or $usingPfx) {
Write-Host "[ERROR] Certificate was provided but signing failed"
$allSigned = $false
# Extract MSI and verify inner exe
$extractDir = Join-Path $env:RUNNER_TEMP "msi-verify"
if (Test-Path $extractDir) { Remove-Item $extractDir -Recurse -Force }
$proc = Start-Process msiexec.exe -ArgumentList '/a', $msiPath, '/qn', "TARGETDIR=$extractDir" -Wait -PassThru -NoNewWindow
if ($proc.ExitCode -eq 0) {
$innerExe = Get-ChildItem -Path $extractDir -Filter "stirling-pdf.exe" -Recurse -File | Select-Object -First 1
if ($innerExe) {
$sig = Get-AuthenticodeSignature -FilePath $innerExe.FullName
Write-Host "Inner EXE: Status=$($sig.Status), Signer=$($sig.SignerCertificate.Subject)"
if ($sig.Status -ne "Valid") {
Write-Host "[ERROR] Inner exe is NOT signed - AV will flag this at runtime"
$allSigned = $false
}
} else {
Write-Host "[INFO] Building unsigned binary (no certificate provided)"
Write-Host "[ERROR] Could not find stirling-pdf.exe inside MSI"
$allSigned = $false
}
} else {
Write-Host "[SUCCESS] MSI is properly signed"
Write-Host "[ERROR] MSI extraction failed (exit code: $($proc.ExitCode))"
$allSigned = $false
}
} else {
Write-Host "[ERROR] MSI not found at $msiPath"
$allSigned = $false
}
if (($usingKeyLocker -or $usingPfx) -and -not $allSigned) {
Write-Host "[ERROR] Code signing verification failed"
if (-not $allSigned) {
Write-Host "[ERROR] Signature verification failed"
exit 1
}
Write-Host "[SUCCESS] MSI and inner exe are properly signed"
- name: Dump smctl logs on failure
if: ${{ failure() && matrix.platform == 'windows-latest' && env.SM_API_KEY != '' }}
shell: pwsh
run: |
$logDir = "$env:USERPROFILE\.signingmanager\logs"
if (Test-Path $logDir) {
Get-ChildItem $logDir | ForEach-Object {
Write-Host "=== $($_.FullName) ==="
Get-Content $_.FullName -Tail 200
Write-Host ""
}
} else {
Write-Host "[SUCCESS] Code signature verification completed"
Write-Host "smctl log directory not found at $logDir"
}
- name: Upload artifacts
@@ -634,8 +480,8 @@ jobs:
fi
else
echo "Checking for Linux artifacts..."
find . -name "*.deb" -o -name "*.AppImage" | head -5
if [ $(find . -name "*.deb" -o -name "*.AppImage" | wc -l) -eq 0 ]; then
find . -name "*.deb" -o -name "*.rpm" -o -name "*.AppImage" | head -5
if [ $(find . -name "*.deb" -o -name "*.rpm" -o -name "*.AppImage" | wc -l) -eq 0 ]; then
echo "❌ No Linux artifacts found"
exit 1
fi
@@ -648,7 +494,7 @@ jobs:
run: |
cd ./frontend/src-tauri/target
echo "Artifact sizes for ${{ matrix.name }}:"
find . -name "*.exe" -o -name "*.dmg" -o -name "*.deb" -o -name "*.AppImage" -o -name "*.msi" | while read file; do
find . -name "*.exe" -o -name "*.dmg" -o -name "*.deb" -o -name "*.rpm" -o -name "*.AppImage" -o -name "*.msi" | while read file; do
if [ -f "$file" ]; then
size=$(stat -c%s "$file" 2>/dev/null || stat -f%z "$file" 2>/dev/null || echo "unknown")
echo "$file: $size bytes"
@@ -692,7 +538,7 @@ jobs:
'Stirling-PDF-windows-x86_64': { icon: '🪟', platform: 'Windows x64', files: '.exe, .msi' },
'Stirling-PDF-macos-aarch64': { icon: '🍎', platform: 'macOS ARM64', files: '.dmg' },
'Stirling-PDF-macos-x86_64': { icon: '🍎', platform: 'macOS Intel', files: '.dmg' },
'Stirling-PDF-linux-x86_64': { icon: '🐧', platform: 'Linux x64', files: '.deb, .AppImage' }
'Stirling-PDF-linux-x86_64': { icon: '🐧', platform: 'Linux x64', files: '.deb, .rpm, .AppImage' }
};
let commentBody = `## 📦 Tauri Desktop Builds Ready!\n\n`;
+2 -2
View File
@@ -167,9 +167,9 @@ jobs:
with:
key: ${{secrets.TESTDRIVER_API_KEY}}
prerun: |
choco install go-task -y
task frontend:build
cd frontend
npm install
npm run build
npm install dashcam-chrome --save
Start-Process "C:/Program Files/Google/Chrome/Application/chrome.exe" -ArgumentList "--start-maximized", "--load-extension=$(pwd)/node_modules/dashcam-chrome/build", "http://${{ secrets.NEW_VPS_HOST }}:1337"
Start-Sleep -Seconds 20
+11
View File
@@ -165,6 +165,7 @@ __pycache__/
# Virtual environments
.env*
!.env*.example
!engine/.env
.venv*
env*/
venv*/
@@ -181,6 +182,7 @@ venv.bak/
.idea/
*.iml
out/
.junie/
# Ignore Mac DS_Store files
.DS_Store
@@ -216,8 +218,14 @@ id_ecdsa.pub
id_ed25519
id_ed25519.pub
.ssh/
# Allow the published GPG release signing public key (safe to share)
!docs/security/signing-key.pub
*ssh
# Taskfile checksum cache
.task/
# cache
.cache
.ruff_cache
@@ -254,3 +262,6 @@ docs/type3/signatures/
# Type3 sample PDFs (development only)
**/type3/samples/
# Claude
.claude/
+1 -1
View File
@@ -16,7 +16,7 @@ repos:
hooks:
- id: codespell
args:
- --ignore-words-list=thirdParty,tabEl,tabEls,Sie,ist
- --ignore-words-list=thirdParty,tabEl,tabEls,Sie,ist,fulfilment
- --skip="./.*,*.csv,*.json,*.ambr"
- --quiet-level=2
files: \.(html|css|js|py|md)$
+117
View File
@@ -0,0 +1,117 @@
version: '3'
tasks:
dev:
desc: "Start backend dev server"
ignore_error: true
cmds:
- cmd: cmd /c gradlew.bat :stirling-pdf:bootRun
platforms: [windows]
- cmd: ./gradlew :stirling-pdf:bootRun
platforms: [linux, darwin]
build:
desc: "Full backend build"
cmds:
- cmd: cmd /c gradlew.bat clean build
platforms: [windows]
- cmd: ./gradlew clean build
platforms: [linux, darwin]
build:fast:
desc: "Build without tests"
cmds:
- cmd: cmd /c gradlew.bat clean build -x test
platforms: [windows]
- cmd: ./gradlew clean build -x test
platforms: [linux, darwin]
build:ci:
desc: "Build for CI (formatting checked separately)"
cmds:
- cmd: cmd /c gradlew.bat build -PnoSpotless
platforms: [windows]
- cmd: ./gradlew build -PnoSpotless
platforms: [linux, darwin]
test:
desc: "Run backend tests"
cmds:
- cmd: cmd /c gradlew.bat test
platforms: [windows]
- cmd: ./gradlew test
platforms: [linux, darwin]
format:
desc: "Auto-fix code formatting"
cmds:
- cmd: cmd /c gradlew.bat spotlessApply
platforms: [windows]
- cmd: ./gradlew spotlessApply
platforms: [linux, darwin]
format:check:
desc: "Check code formatting"
cmds:
- cmd: cmd /c gradlew.bat spotlessCheck
platforms: [windows]
- cmd: ./gradlew spotlessCheck
platforms: [linux, darwin]
fix:
desc: "Auto-fix backend"
cmds:
- task: format
swagger:
desc: "Generate OpenAPI docs"
cmds:
- cmd: cmd /c gradlew.bat :stirling-pdf:copySwaggerDoc
platforms: [windows]
- cmd: ./gradlew :stirling-pdf:copySwaggerDoc
platforms: [linux, darwin]
sources:
- app/core/src/main/java/**/*.java
- app/proprietary/src/main/java/**/*.java
- app/common/src/main/java/**/*.java
generates:
- SwaggerDoc.json
check:
desc: "Backend quality gate"
cmds:
- task: format:check
- task: test
version:
desc: "Print project version"
silent: true
cmds:
- cmd: cmd /c gradlew.bat printVersion --quiet | tail -1
platforms: [windows]
- cmd: ./gradlew printVersion --quiet | tail -1
platforms: [linux, darwin]
licenses:check:
desc: "Check dependency licenses"
cmds:
- cmd: cmd /c gradlew.bat checkLicense --no-parallel
platforms: [windows]
- cmd: ./gradlew checkLicense --no-parallel
platforms: [linux, darwin]
licenses:generate:
desc: "Check and generate dependency license report"
cmds:
- cmd: cmd /c gradlew.bat checkLicense generateLicenseReport --no-parallel
platforms: [windows]
- cmd: ./gradlew checkLicense generateLicenseReport --no-parallel
platforms: [linux, darwin]
clean:
desc: "Clean build artifacts"
cmds:
- cmd: cmd /c gradlew.bat clean
platforms: [windows]
- cmd: ./gradlew clean
platforms: [linux, darwin]
+105
View File
@@ -0,0 +1,105 @@
version: '3'
vars:
JLINK_MODULES: "java.base,java.compiler,java.desktop,java.instrument,java.logging,java.management,java.naming,java.net.http,java.prefs,java.rmi,java.scripting,java.security.jgss,java.security.sasl,java.sql,java.transaction.xa,java.xml,java.xml.crypto,jdk.crypto.ec,jdk.crypto.cryptoki,jdk.unsupported"
tasks:
prepare:
desc: "Prepare desktop build dependencies"
deps: [jlink, ":frontend:prepare:desktop", provisioner]
provisioner:
desc: "Build installer provisioner"
platforms: [windows]
cmds:
- node scripts/build-provisioner.mjs
dev:
desc: "Start Tauri desktop dev mode"
deps: [prepare]
ignore_error: true
cmds:
- npx tauri dev --no-watch
build:
desc: "Build Tauri desktop app (production)"
deps: [prepare]
cmds:
- npx tauri build
build:dev:
desc: "Build Tauri desktop app (dev, no bundling)"
deps: [prepare]
cmds:
- npx tauri build --no-bundle
build:dev:mac:
desc: "Build Tauri desktop .app bundle (macOS)"
deps: [prepare]
cmds:
- npx tauri build --bundles app
build:dev:windows:
desc: "Build Tauri desktop NSIS installer (Windows)"
deps: [prepare]
cmds:
- npx tauri build --bundles nsis
build:dev:linux:
desc: "Build Tauri desktop AppImage (Linux)"
deps: [prepare]
cmds:
- npx tauri build --bundles appimage
clean:
desc: "Clean Tauri/Cargo build artifacts"
cmds:
- task: jlink:clean
- cd src-tauri && cargo clean
- rm -rf dist build
# ============================================================
# JLink — Build bundled Java runtime for Tauri
# ============================================================
jlink:
desc: "Build backend JAR and create JLink runtime for Tauri"
deps: [jlink:jar, jlink:runtime]
jlink:jar:
desc: "Build backend JAR for Tauri bundling"
run: once
dir: ..
env:
DISABLE_ADDITIONAL_FEATURES: "true"
cmds:
- cmd: cmd /c gradlew.bat bootJar --no-daemon
platforms: [windows]
- cmd: ./gradlew bootJar --no-daemon
platforms: [linux, darwin]
- mkdir -p frontend/src-tauri/libs
- cp app/core/build/libs/stirling-pdf-*.jar frontend/src-tauri/libs/
status:
- test -f frontend/src-tauri/libs/stirling-pdf-*.jar
jlink:runtime:
desc: "Create custom JRE with jlink"
deps: [jlink:jar]
cmds:
- rm -rf src-tauri/runtime/jre
- mkdir -p src-tauri/runtime
- >-
jlink
--add-modules {{.JLINK_MODULES}}
--strip-debug
--compress=2
--no-header-files
--no-man-pages
--output src-tauri/runtime/jre
status:
- test -d src-tauri/runtime/jre
jlink:clean:
desc: "Remove JLink runtime and bundled JARs"
cmds:
- rm -rf src-tauri/libs src-tauri/runtime
+57
View File
@@ -0,0 +1,57 @@
version: '3'
vars:
COMPOSE_DIR: docker/compose
EMBEDDED_DIR: docker/embedded
tasks:
build:
desc: "Build standard Docker image"
cmds:
- docker build -t stirling-pdf -f {{.EMBEDDED_DIR}}/Dockerfile .
build:fat:
desc: "Build fat Docker image (all features)"
cmds:
- docker build -t stirling-pdf-fat -f {{.EMBEDDED_DIR}}/Dockerfile.fat .
build:ultra-lite:
desc: "Build ultra-lite Docker image"
cmds:
- docker build -t stirling-pdf-ultra-lite -f {{.EMBEDDED_DIR}}/Dockerfile.ultra-lite .
build:frontend:
desc: "Build frontend-only Docker image"
cmds:
- docker build -t stirling-pdf-frontend -f docker/frontend/Dockerfile .
build:engine:
desc: "Build engine Docker image"
dir: engine
cmds:
- docker build -t stirling-pdf-engine .
up:
desc: "Start standard docker compose stack"
cmds:
- docker compose -f {{.COMPOSE_DIR}}/docker-compose.yml up -d
up:fat:
desc: "Start fat docker compose stack"
cmds:
- docker compose -f {{.COMPOSE_DIR}}/docker-compose.fat.yml up -d
up:ultra-lite:
desc: "Start ultra-lite docker compose stack"
cmds:
- docker compose -f {{.COMPOSE_DIR}}/docker-compose.ultra-lite.yml up -d
down:
desc: "Stop all running docker compose stacks"
cmds:
- docker compose -f {{.COMPOSE_DIR}}/docker-compose.yml down
logs:
desc: "Tail docker compose logs"
cmds:
- docker compose -f {{.COMPOSE_DIR}}/docker-compose.yml logs -f
+127
View File
@@ -0,0 +1,127 @@
version: '3'
tasks:
install:
desc: "Install engine dependencies"
run: once
cmds:
- uv python install 3.13.8
- uv sync
sources:
- uv.lock
- pyproject.toml
status:
- test -d .venv
prepare:
desc: "Set up engine .env from template"
deps: [install]
cmds:
- uv run scripts/setup_env.py
sources:
- scripts/setup_env.py
generates:
- .env.local
run:
desc: "Run engine server"
deps: [prepare]
ignore_error: true
dir: src
env:
PYTHONUNBUFFERED: "1"
cmds:
- uv run uvicorn stirling.api.app:app --host 0.0.0.0 --port 5001
dev:
desc: "Start engine dev server with hot reload"
deps: [prepare]
ignore_error: true
dir: src
env:
PYTHONUNBUFFERED: "1"
cmds:
- uv run uvicorn stirling.api.app:app --host 0.0.0.0 --port 5001 --reload
lint:
desc: "Run linting"
deps: [install]
cmds:
- uv run ruff check .
lint:fix:
desc: "Auto-fix lint issues"
deps: [install]
cmds:
- uv run ruff check . --fix
format:
desc: "Auto-fix code formatting"
deps: [install]
cmds:
- uv run ruff format .
format:check:
desc: "Check code formatting"
deps: [install]
cmds:
- uv run ruff format . --diff
typecheck:
desc: "Run type checking"
deps: [install]
cmds:
- uv run pyright . --warnings
test:
desc: "Run tests"
deps: [prepare]
cmds:
- uv run pytest tests
fix:
desc: "Auto-fix lint + format"
cmds:
- task: lint:fix
- task: format
check:
desc: "Full engine quality gate"
cmds:
- task: typecheck
- task: lint
- task: format:check
- task: test
tool-models:
desc: "Generate tool_models.py from Java OpenAPI spec (SwaggerDoc.json)"
deps: [install, ":backend:swagger"]
cmds:
- uv run python scripts/generate_tool_models.py --spec ../SwaggerDoc.json --output src/stirling/models/tool_models.py
sources:
- ../SwaggerDoc.json
- scripts/generate_tool_models.py
generates:
- src/stirling/models/tool_models.py
clean:
desc: "Clean build artifacts"
cmds:
- task: '{{if eq .OS "Windows_NT"}}clean-windows{{else}}clean-unix{{end}}'
clean-unix:
internal: true
desc: "Clean build artifacts"
cmds:
- rm -rf .venv data logs output
# On Windows, use PowerShell as bash failed to delete some dependencies
clean-windows:
internal: true
desc: "Clean build artifacts"
ignore_error: true
cmds:
- powershell rm -Recurse -Force -ErrorAction SilentlyContinue .venv
- powershell rm -Recurse -Force -ErrorAction SilentlyContinue data
- powershell rm -Recurse -Force -ErrorAction SilentlyContinue logs
- powershell rm -Recurse -Force -ErrorAction SilentlyContinue output
+313
View File
@@ -0,0 +1,313 @@
version: '3'
tasks:
install:
desc: "Install dependencies"
run: once
cmds:
- '{{ if eq .CI "true" }}npm ci{{ else }}npm install{{ end }}'
sources:
- package-lock.json
- package.json
status:
- test -d node_modules
env:
CI: '{{ .CI | default "false" }}'
prepare:env:
desc: "Generate .env from example if missing"
run: once
deps: [install]
cmds:
- npx tsx scripts/setup-env.ts
sources:
- scripts/setup-env.ts
- config/.env.example
generates:
- .env
prepare:env:saas:
desc: "Generate .env and .env.saas from examples if missing"
run: once
deps: [install]
cmds:
- npx tsx scripts/setup-env.ts --saas
sources:
- scripts/setup-env.ts
- config/.env.example
- config/.env.saas.example
generates:
- .env
- .env.saas
prepare:env:desktop:
desc: "Generate .env and .env.desktop from examples if missing"
run: once
deps: [install]
cmds:
- npx tsx scripts/setup-env.ts --desktop
sources:
- scripts/setup-env.ts
- config/.env.example
- config/.env.desktop.example
generates:
- .env
- .env.desktop
prepare:icons:
desc: "Generate icon bundle from source references"
run: once
deps: [install]
cmds:
- node scripts/generate-icons.js
prepare:
desc: "Set up dev environment"
run: once
deps: [prepare:env, prepare:icons]
prepare:saas:
desc: "Prepare for SaaS mode"
run: once
deps: [prepare:env:saas, prepare:icons]
prepare:desktop:
desc: "Prepare for desktop mode"
run: once
deps: [prepare:env:desktop, prepare:icons]
# ============================================================
# Development
# ============================================================
dev:
desc: "Start frontend dev server"
deps: [prepare]
ignore_error: true
cmds:
- npx vite
dev:core:
desc: "Start frontend dev server in core mode"
deps: [prepare]
ignore_error: true
cmds:
- npx vite --mode core
dev:proprietary:
desc: "Start frontend dev server in proprietary mode"
deps: [prepare]
ignore_error: true
cmds:
- npx vite --mode proprietary
dev:saas:
desc: "Start frontend dev server in SaaS mode"
deps: [prepare:saas]
ignore_error: true
cmds:
- npx vite --mode saas
dev:desktop:
desc: "Start frontend dev server in desktop mode"
deps: [prepare:desktop]
ignore_error: true
cmds:
- npx vite --mode desktop
dev:prototypes:
desc: "Start frontend dev server in prototypes mode"
deps: [prepare]
ignore_error: true
cmds:
- npx vite --mode prototypes
# ============================================================
# Build
# ============================================================
build:
desc: "Production build (default mode)"
deps: [prepare]
cmds:
- npx vite build
build:core:
desc: "Build for core mode"
deps: [prepare]
cmds:
- npx vite build --mode core
build:proprietary:
desc: "Build for proprietary mode"
deps: [prepare]
cmds:
- npx vite build --mode proprietary
build:saas:
desc: "Build for SaaS mode"
deps: [prepare:saas]
cmds:
- npx vite build --mode saas
build:desktop:
desc: "Build for desktop mode"
deps: [prepare:desktop]
cmds:
- npx vite build --mode desktop
build:prototypes:
desc: "Build for prototypes mode"
deps: [prepare]
cmds:
- npx vite build --mode prototypes
# ============================================================
# Code quality
# ============================================================
lint:
desc: "Run linting"
deps: [install]
cmds:
- npx eslint --max-warnings=0
- npx dpdm src --circular --no-warning --no-tree --exit-code circular:1
lint:fix:
desc: "Auto-fix lint issues"
deps: [install]
cmds:
- npx eslint --fix
format:
desc: "Auto-fix code formatting"
deps: [install]
cmds:
- npx prettier --write .
format:check:
desc: "Check code formatting"
deps: [install]
cmds:
- npx prettier --check .
fix:
desc: "Auto-fix lint and format"
cmds:
- task: format
- task: lint:fix
typecheck:
desc: "Typecheck default build of the app"
cmds:
- task: typecheck:proprietary
typecheck:core:
desc: "Typecheck core build variant"
deps: [prepare]
cmds:
- npx tsc --noEmit --project src/core/tsconfig.json
typecheck:proprietary:
desc: "Typecheck proprietary build variant"
deps: [prepare]
cmds:
- npx tsc --noEmit --project src/proprietary/tsconfig.json
typecheck:saas:
desc: "Typecheck SaaS build variant"
deps: [prepare:saas]
cmds:
- npx tsc --noEmit --project src/saas/tsconfig.json
typecheck:desktop:
desc: "Typecheck desktop build variant"
deps: [prepare:desktop]
cmds:
- npx tsc --noEmit --project src/desktop/tsconfig.json
typecheck:scripts:
desc: "Typecheck scripts"
deps: [prepare]
cmds:
- npx tsc --noEmit --project scripts/tsconfig.json
typecheck:prototypes:
desc: "Typecheck prototypes build variant"
deps: [prepare]
cmds:
- npx tsc --noEmit --project src/prototypes/tsconfig.json
typecheck:all:
desc: "Typecheck all build variants"
cmds:
- task: typecheck:core
- task: typecheck:proprietary
- task: typecheck:saas
- task: typecheck:desktop
- task: typecheck:scripts
# ============================================================
# Quality Gate
# ============================================================
check:
desc: "Quick quality gate for local development"
cmds:
- task: typecheck
- task: lint
- task: format:check
- task: test
check:all:
desc: "Full CI quality gate"
cmds:
- task: typecheck:all
- task: lint
- task: format:check
- task: build
- task: test
# ============================================================
# Test
# ============================================================
test:
desc: "Run tests"
deps: [install]
cmds:
- npx vitest run
test:watch:
desc: "Run tests in watch mode"
deps: [install]
cmds:
- npx vitest --watch
test:coverage:
desc: "Run tests with coverage"
deps: [install]
cmds:
- npx vitest --coverage
test:e2e:
desc: "Run E2E tests"
deps: [prepare]
cmds:
- npx playwright test {{.CLI_ARGS}}
test:e2e:install:
desc: "Install E2E test browsers"
deps: [install]
cmds:
- npx playwright install {{.CLI_ARGS}} --with-deps
# ============================================================
# Code Generation
# ============================================================
licenses:generate:
desc: "Generate frontend license report"
deps: [install]
cmds:
- node scripts/generate-licenses.js
+52 -24
View File
@@ -2,18 +2,42 @@
This file provides guidance to AI Agents when working with code in this repository.
## Taskfile (Recommended)
This project uses [Task](https://taskfile.dev/) as a unified command runner. All build, dev, test, lint, and docker commands can be run from the repo root via `task <command>`. Run `task --list` to see all available commands.
### Quick Reference
- `task install` — install all dependencies
- `task dev` — start backend + frontend concurrently
- `task dev:all` — start backend + frontend + engine concurrently
- `task build` — build all components
- `task test` — run all tests (backend + frontend + engine)
- `task lint` — run all linters
- `task format` — auto-fix formatting across all components
- `task check` — full quality gate (lint + typecheck + test)
- `task clean` — clean all build artifacts
- `task docker:build` — build standard Docker image
- `task docker:up` — start Docker compose stack
## Common Development Commands
### Build and Test
- **Build project**: `./gradlew clean build`
- **Run locally**: `./gradlew bootRun`
- **Full test suite**: `./test.sh` (builds all Docker variants and runs comprehensive tests)
- **Code formatting**: `./gradlew spotlessApply` (runs automatically before compilation)
- **Build project**: `task build`
- **Run backend locally**: `task backend:dev`
- **Run all tests**: `task test` (or individually: `task backend:test`, `task frontend:test`, `task engine:test`)
- **Docker integration tests**: `./test.sh` (builds all Docker variants and runs comprehensive tests)
- **Code formatting**: `task format` (or `task backend:format` for Java only)
- **Full quality gate**: `task check` (runs lint + typecheck + test across all components)
After modifying any files in the project, you must run the relevant `task check` command that covers that area of the code. For example, when editing frontend files run `task frontend:check`; for Python engine files run `task engine:check`; for Java backend files run `task backend:check`.
### Docker Development
- **Build ultra-lite**: `docker build -t stirlingtools/stirling-pdf:latest-ultra-lite -f ./Dockerfile.ultra-lite .`
- **Build standard**: `docker build -t stirlingtools/stirling-pdf:latest -f ./Dockerfile .`
- **Build fat version**: `docker build -t stirlingtools/stirling-pdf:latest-fat -f ./Dockerfile.fat .`
- **Build standard**: `task docker:build` (or `docker build -t stirling-pdf -f docker/embedded/Dockerfile .`)
- **Build fat version**: `task docker:build:fat`
- **Build ultra-lite**: `task docker:build:ultra-lite`
- **Start compose stack**: `task docker:up` (or `task docker:up:fat`, `task docker:up:ultra-lite`)
- **Stop compose stack**: `task docker:down`
- **View logs**: `task docker:logs`
- **Example compose files**: Located in `exampleYmlFiles/` directory
### Security Mode Development
@@ -23,20 +47,22 @@ Set `DOCKER_ENABLE_SECURITY=true` environment variable to enable security featur
Development for the AI engine happens in the `engine/` folder. The frontend calls the Python via Java as a proxy.
- Follow the engine-specific guidance in [engine/AGENTS.md](engine/AGENTS.md) for Python architecture, code style, and AI usage.
- Use Makefile commands for Python work:
- From `engine/`: `make check` to lint, type-check, test, etc. and `make fix` to fix easily fixable linting and formatting issues.
- The project structure is defined in `engine/pyproject.toml`. Any new dependencies should be listed there, followed by running `make install`.
- Use Task commands from the repo root:
- `task engine:check` lint, type-check, test
- `task engine:fix` — auto-fix linting and formatting
- `task engine:install` — install dependencies
- The project structure is defined in `engine/pyproject.toml`. Any new dependencies should be listed there, followed by running `task engine:install`.
### Frontend Development
- **Frontend dev server**: `cd frontend && npm run dev` (requires backend on localhost:8080)
- **Frontend dev server**: `task frontend:dev` requires backend on localhost:8080
- **Tech Stack**: Vite + React + TypeScript + Mantine UI + TailwindCSS
- **Proxy Configuration**: Vite proxies `/api/*` calls to backend (localhost:8080)
- **Build Process**: DO NOT run build scripts manually - builds are handled by CI/CD pipelines
- **Package Installation**: DO NOT run npm install commands - package management handled separately
- **Package Installation**: `task frontend:install`
- **Deployment Options**:
- **Desktop App**: `npm run tauri-build` (native desktop application)
- **Web Server**: `npm run build` then serve dist/ folder
- **Development**: `npm run tauri-dev` for desktop dev mode
- **Desktop App**: `task desktop:build`
- **Web Server**: `task frontend:build` then serve dist/ folder
- **Development**: `task desktop:dev` for desktop dev mode
#### Environment Variables
- All `VITE_*` variables must be declared in the appropriate example file:
@@ -44,8 +70,8 @@ Development for the AI engine happens in the `engine/` folder. The frontend call
- `frontend/config/.env.saas.example` — SaaS-only vars
- `frontend/config/.env.desktop.example` — desktop (Tauri)-only vars
- Never use `|| 'hardcoded-fallback'` inline — put defaults in the example files
- `npm run prep` / `prep:saas` / `prep:desktop` auto-create the env files from examples on first run, and error if any required keys are missing
- These prep scripts run automatically at the start of all `dev*`, `build*`, and `tauri*` commands
- `task frontend:prepare` / `prepare:saas` / `prepare:desktop` auto-create the env files from examples on first run, and error if any required keys are missing
- Prepare runs automatically as a dependency of all `dev*`, `build*`, and `desktop*` tasks
- See `frontend/README.md#environment-variables` for full documentation
#### Import Paths - CRITICAL
@@ -299,15 +325,17 @@ The frontend is organized with a clear separation of concerns:
## Development Workflow
1. **Local Development**:
- Backend: `./gradlew bootRun` (runs on localhost:8080)
- Frontend: `cd frontend && npm run dev` (runs on localhost:5173, proxies to backend)
2. **Docker Testing**: Use `./test.sh` before submitting PRs
3. **Code Style**: Spotless enforces Google Java Format automatically
4. **Translations**:
1. **Local Development** (using Taskfile):
- Backend + frontend: `task dev`
- All services (including AI engine): `task dev:all`
- Or individually: `task backend:dev` (localhost:8080), `task frontend:dev` (localhost:5173), `task engine:dev` (localhost:5001)
2. **Quality Gate**: Run `task check` before submitting PRs
3. **Docker Testing**: Use `./test.sh` for full Docker integration tests
4. **Code Style**: Spotless enforces Google Java Format automatically (`task backend:format`)
5. **Translations**:
- Backend: Use helper scripts in `/scripts` for multi-language updates
- Frontend: Update JSON files in `frontend/public/locales/` or use conversion script
5. **Documentation**: API docs auto-generated and available at `/swagger-ui/index.html`
6. **Documentation**: API docs auto-generated and available at `/swagger-ui/index.html`
## Frontend Architecture Status
+16 -2
View File
@@ -15,6 +15,19 @@ Before you start working on an issue, please comment on (or create) the issue an
Once you have been assigned an issue, you can start working on it. When you are ready to submit your changes, open a pull request.
For a detailed pull request tutorial, see [this guide](https://www.digitalocean.com/community/tutorials/how-to-create-a-pull-request-on-github).
## Development Quick Start
This project uses [Task](https://taskfile.dev/) as a unified command runner. After cloning:
1. Install the `task` CLI: https://taskfile.dev/installation/
2. Run `task install` to install all dependencies
3. Run `task dev` to start backend + frontend
4. Run `task check` before submitting a PR
Run `task --list` to see all available commands.
## Pull Request Guidelines
Please make sure your Pull Request adheres to the following guidelines:
- Use the PR template provided.
@@ -39,9 +52,10 @@ If, at any point in time, you have a question, please feel free to ask in the sa
## Developer Documentation
For technical guides, setup instructions, and development resources, please see our [Developer Documentation](devGuide/) which includes:
For technical guides, setup instructions, and development resources:
- [Developer Guide](devGuide/DeveloperGuide.md) - Main setup and architecture guide
- [Developer Guide](DeveloperGuide.md) - Main setup and architecture guide
- [Taskfile.yml](Taskfile.yml) - Unified task runner for all build/dev/test/lint commands
- [Exception Handling Guide](devGuide/EXCEPTION_HANDLING_GUIDE.md) - Error handling patterns and i18n
- [Translation Guide](devGuide/HowToAddNewLanguage.md) - Adding new languages
- And more in the [devGuide folder](devGuide/)
+59 -13
View File
@@ -42,11 +42,13 @@ This guide focuses on developing for Stirling 2.0, including both the React fron
### Prerequisites
- [Task](https://taskfile.dev/installation/) — unified command runner (recommended)
- Docker
- Git
- Java JDK 21 or later (JDK 25 recommended)
- Node.js 18+ and npm (required for frontend development)
- Gradle 7.0 or later (Included within the repo)
- [uv](https://docs.astral.sh/uv/) — Python package manager (required for engine development)
- Rust and Cargo (required for Tauri desktop app development)
- Tauri CLI (install with `cargo install tauri-cli`)
@@ -82,13 +84,29 @@ For local testing, you should generally be testing the full 'Security' version o
5. **Frontend Setup (Required for Stirling 2.0)**
Navigate to the frontend directory and install dependencies using npm.
### Verify Setup
Run `task install` to install all project dependencies (frontend npm packages, engine Python packages). Gradle manages its own dependencies automatically. Then run `task check` to verify everything builds and passes.
## 4. Stirling 2.0 Development Workflow
### Using Taskfile (Recommended)
The fastest way to start developing:
1. **Start developing**: `task dev` (runs backend + frontend concurrently — Ctrl+C to stop)
2. **Or start services individually** in separate terminals:
- `task backend:dev` — Spring Boot on localhost:8080
- `task frontend:dev` — Vite on localhost:5173
- `task engine:dev` — FastAPI on localhost:5001
Run `task --list` to see all available commands.
### Frontend Development (React)
The frontend is a React SPA that runs independently during development:
1. **Start the backend**: Run the Spring Boot application (serves API endpoints on localhost:8080)
2. **Start the frontend dev server**: Navigate to the frontend directory and run the development server (serves UI on localhost:5173)
1. **Start the backend**: `task backend:dev` (serves API endpoints on localhost:8080)
2. **Start the frontend dev server**: `task frontend:dev` (serves UI on localhost:5173)
3. **Development flow**: The Vite dev server automatically proxies API calls to the backend
### File Storage Architecture
@@ -99,7 +117,10 @@ Stirling 2.0 uses client-side file storage:
### Tauri Desktop App Development
Stirling-PDF can be packaged as a cross-platform desktop application using Tauri with PDF file association support and bundled JRE.
See [the frontend README](frontend/README.md#tauri) for build instructions.
Using Taskfile: `task desktop:dev` (development) or `task desktop:build` (production build).
See [the frontend README](frontend/README.md#tauri) for detailed build instructions.
## 5. Project Structure
@@ -187,7 +208,7 @@ services:
limits:
memory: 4G
healthcheck:
test: ["CMD-SHELL", "curl -f http://localhost:8080/api/v1/info/status | grep -q 'UP' && curl -fL http://localhost:8080/ | grep -q 'Please sign in'"]
test: ["CMD-SHELL", "curl -f http://localhost:8080$${SYSTEM_ROOTURIPATH:-''}/api/v1/info/status | grep -q 'UP' && curl -fL http://localhost:8080/ | grep -q 'Please sign in'"]
interval: 5s
timeout: 10s
retries: 16
@@ -222,6 +243,20 @@ docker-compose -f exampleYmlFiles/docker-compose-latest-security.yml up
### Building Docker Images
#### Using Taskfile (Recommended)
```bash
task docker:build # standard image
task docker:build:fat # fat image (all features)
task docker:build:ultra-lite # ultra-lite image
task docker:up # start standard compose stack
task docker:up:fat # start fat compose stack
task docker:down # stop all stacks
task docker:logs # tail logs
```
#### Manual Docker Builds
Stirling-PDF uses different Docker images for various configurations. The build process is controlled by environment variables and uses specific Dockerfile variants. Here's how to build the Docker images:
1. Set the security environment variable:
@@ -230,10 +265,10 @@ Stirling-PDF uses different Docker images for various configurations. The build
export DISABLE_ADDITIONAL_FEATURES=true # or false for to enable login and security features for builds
```
2. Build the project with Gradle:
2. Build the project:
```bash
./gradlew clean build
task backend:build
```
3. Build the Docker images:
@@ -261,9 +296,18 @@ Note: The `--no-cache` and `--pull` flags ensure that the build process uses the
## 7. Testing
### Quick Testing with Taskfile
Run all unit/integration tests across all components:
```bash
task test # run all tests (backend + frontend + engine)
task check # full quality gate: lint + typecheck + test
```
### Comprehensive Testing Script
Stirling-PDF provides a `test.sh` script in the root directory. This script builds all versions of Stirling-PDF, checks that each version works, and runs Cucumber tests. It's recommended to run this script before submitting a final pull request.
Stirling-PDF also provides a `test.sh` script in the root directory for Docker integration tests. This script builds all versions of Stirling-PDF, checks that each version works, and runs Cucumber tests. It's recommended to run this script before submitting a final pull request.
To run the test script:
@@ -289,10 +333,11 @@ Note: The `test.sh` script will run automatically when you raise a PR. However,
For React frontend development:
1. Start the backend: Run the Spring Boot application to serve API endpoints on localhost:8080
2. Start the frontend dev server: Navigate to the frontend directory and run the development server on localhost:5173
1. Start the backend: `task backend:dev` (serves API endpoints on localhost:8080)
2. Start the frontend dev server: `task frontend:dev` (serves UI on localhost:5173)
3. The Vite dev server automatically proxies API calls to the backend
4. Test React components, UI interactions, and IndexedDB file operations using browser developer tools
4. Run frontend tests: `task frontend:test` (or `task frontend:test:watch` for watch mode)
5. Test React components, UI interactions, and IndexedDB file operations using browser developer tools
### Local Testing (Java and UI Components)
@@ -308,7 +353,7 @@ To run Stirling-PDF locally:
1. Compile and run the project using built-in IDE methods or by running:
```bash
./gradlew bootRun
task backend:dev
```
2. Access the application at `http://localhost:8080` in your web browser.
@@ -329,10 +374,11 @@ Important notes:
2. Create a new branch for your feature or bug fix.
3. Make your changes and commit them with clear, descriptive messages and ensure any documentation is updated related to your changes.
4. Test your changes thoroughly in the Docker environment.
5. Run the `test.sh` script to ensure all versions build correctly and pass the Cucumber tests:
5. Run the quality gate and integration tests:
```bash
./test.sh
task check # lint + typecheck + test across all components
./test.sh # Docker integration tests (builds all variants + Cucumber)
```
6. Push your changes to your fork.
+2
View File
@@ -14,6 +14,8 @@ if that directory exists, is licensed under the license defined in "frontend/src
if that directory exists, is licensed under the license defined in "frontend/src/desktop/LICENSE".
* All content that resides under the "frontend/src/saas/" directory of this repository,
if that directory exists, is licensed under the license defined in "frontend/src/saas/LICENSE".
* All content that resides under the "frontend/src/prototypes/" directory of this repository,
if that directory exists, is licensed under the license defined in "frontend/src/prototypes/LICENSE".
* Content outside of the above mentioned directories or restrictions above is
available under the MIT License as defined below.
+1 -1
View File
@@ -60,7 +60,7 @@ For full installation options (including desktop and Kubernetes), see our [Docum
We welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
For development setup, see the [Developer Guide](DeveloperGuide.md).
This project uses [Task](https://taskfile.dev/) as a unified command runner for all build, dev, and test commands. Run `task install` to get started, or see the [Developer Guide](DeveloperGuide.md) for full details.
For adding translations, see the [Translation Guide](devGuide/HowToAddNewLanguage.md).
+128
View File
@@ -0,0 +1,128 @@
version: '3'
output: prefixed
includes:
backend:
taskfile: .taskfiles/backend.yml
dir: .
frontend:
taskfile: .taskfiles/frontend.yml
dir: frontend
engine:
taskfile: .taskfiles/engine.yml
dir: engine
docker:
taskfile: .taskfiles/docker.yml
dir: .
desktop:
taskfile: .taskfiles/desktop.yml
dir: frontend
tasks:
# ============================================================
# Setup & Prerequisites
# ============================================================
install:
desc: "Install all project dependencies"
cmds:
- task: frontend:install
- task: engine:install
# ============================================================
# Development
# ============================================================
dev:
desc: "Start backend + frontend concurrently"
deps:
- backend:dev
- frontend:dev
dev:all:
desc: "Start backend + frontend + engine concurrently"
deps:
- backend:dev
- frontend:dev:prototypes
- engine:dev
# ============================================================
# Build
# ============================================================
build:
desc: "Build all components"
cmds:
- task: backend:build
- task: frontend:build
# ============================================================
# Test
# ============================================================
test:
desc: "Run ALL tests (backend + frontend + engine)"
cmds:
- task: backend:test
- task: frontend:test
- task: engine:test
# ============================================================
# Lint & Format
# ============================================================
lint:
desc: "Run all linters"
cmds:
- task: frontend:lint
- task: engine:lint
fix:
desc: "Auto-fix all components"
cmds:
- task: backend:fix
- task: frontend:fix
- task: engine:fix
format:
desc: "Auto-fix formatting across all components"
cmds:
- task: backend:format
- task: frontend:format
- task: engine:format
format:check:
desc: "Check formatting across all components"
cmds:
- task: backend:format:check
- task: frontend:format:check
- task: engine:format:check
# ============================================================
# Quality Gate
# ============================================================
check:
desc: "Quick quality gate for local development"
cmds:
- task: backend:check
- task: frontend:check
- task: engine:check
check:all:
desc: "Full CI quality gate"
cmds:
- task: backend:check
- task: frontend:check:all
- task: engine:check
# ============================================================
# Clean
# ============================================================
clean:
desc: "Clean all build artifacts"
cmds:
- task: backend:clean
- task: engine:clean
+5 -3
View File
@@ -7,6 +7,8 @@ spotless {
target 'src/**/java/**/*.java'
targetExclude 'src/main/java/org/apache/**'
googleJavaFormat(googleJavaFormatVersion).aosp().reorderImports(false)
// google-java-format 1.28.0 bundles Guava 32.x which crashes Spotless lint on JDK 24/25
suppressLintsFor { setStep('google-java-format') }
importOrder("java", "javax", "org", "com", "net", "io", "jakarta", "lombok", "me", "stirling")
trimTrailingWhitespace()
@@ -27,10 +29,10 @@ spotless {
}
}
dependencies {
api 'com.google.guava:guava:33.4.8-jre'
api 'com.google.guava:guava:33.5.0-jre'
api 'org.springframework.boot:spring-boot-starter-webmvc'
api 'org.springframework.boot:spring-boot-starter-aspectj'
api 'com.googlecode.owasp-java-html-sanitizer:owasp-java-html-sanitizer:20260102.1'
api 'com.googlecode.owasp-java-html-sanitizer:owasp-java-html-sanitizer:20260313.1'
api 'com.fathzer:javaluator:3.0.6'
api 'com.posthog.java:posthog:1.2.0'
api 'org.apache.commons:commons-lang3:3.20.0'
@@ -43,7 +45,7 @@ dependencies {
api 'com.github.junrar:junrar:7.5.8' // RAR archive support for CBR files
api 'jakarta.servlet:jakarta.servlet-api:6.1.0'
api 'org.snakeyaml:snakeyaml-engine:3.0.1'
api "org.springdoc:springdoc-openapi-starter-webmvc-ui:3.0.1"
api "org.springdoc:springdoc-openapi-starter-webmvc-ui:3.0.2"
// Simple Java Mail for EML/MSG parsing (replaces direct Angus Mail usage)
api 'org.simplejavamail:simple-java-mail:8.12.6'
api 'org.simplejavamail:outlook-module:8.12.6' // MSG file support
@@ -75,6 +75,7 @@ public class ApplicationProperties {
private AutoPipeline autoPipeline = new AutoPipeline();
private ProcessExecutor processExecutor = new ProcessExecutor();
private PdfEditor pdfEditor = new PdfEditor();
private AiEngine aiEngine = new AiEngine();
@Bean
public PropertySource<?> dynamicYamlPropertySource(ConfigurableEnvironment environment)
@@ -231,6 +232,13 @@ public class ApplicationProperties {
}
}
@Data
public static class AiEngine {
private boolean enabled = false;
private String url = "http://localhost:5001";
private int timeoutSeconds = 120;
}
@Data
public static class Legal {
private String termsAndConditions;
@@ -1,8 +1,10 @@
package stirling.software.common.service;
import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.UUID;
@@ -10,6 +12,7 @@ import java.util.UUID;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
@@ -143,6 +146,24 @@ public class FileStorage {
return new StoredFile(fileId, size);
}
public String storeFromStreamingBody(StreamingResponseBody body, String originalName)
throws IOException {
String fileId = generateFileId();
Path filePath = getFilePath(fileId);
Files.createDirectories(filePath.getParent());
boolean success = false;
try (OutputStream os = new BufferedOutputStream(Files.newOutputStream(filePath))) {
body.writeTo(os);
success = true;
} finally {
if (!success) {
Files.deleteIfExists(filePath);
}
}
log.debug("Stored StreamingResponseBody with ID: {}", fileId);
return fileId;
}
/**
* Delete a file by its ID
*
@@ -0,0 +1,184 @@
package stirling.software.common.service;
import java.io.IOException;
import java.io.UncheckedIOException;
import java.net.URLDecoder;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.util.regex.Pattern;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.core.env.Environment;
import org.springframework.core.io.FileSystemResource;
import org.springframework.core.io.Resource;
import org.springframework.http.*;
import org.springframework.stereotype.Service;
import org.springframework.util.MultiValueMap;
import org.springframework.web.client.RequestCallback;
import org.springframework.web.client.RestTemplate;
import jakarta.servlet.ServletContext;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.model.enumeration.Role;
import stirling.software.common.util.TempFile;
import stirling.software.common.util.TempFileManager;
/**
* Dispatches HTTP POST requests to internal Stirling API endpoints via loopback. Used by
* PipelineProcessor and AiWorkflowService to execute tool operations programmatically without
* leaving the JVM network stack.
*/
@Service
@Slf4j
public class InternalApiClient {
// Allowlist for internal dispatch. Matches a fixed namespace prefix,
// but rejects traversal (..), URL-encoding (%), query/fragment, backslashes, and any other
// character that could alter the resolved endpoint on the local Spring server.
private static final Pattern ALLOWED_ENDPOINT_PATH =
Pattern.compile("^/api/v1/(general|misc|security|convert|filter)(/[A-Za-z0-9_-]+)+$");
private final ServletContext servletContext;
private final UserServiceInterface userService;
private final TempFileManager tempFileManager;
private final Environment environment;
public InternalApiClient(
ServletContext servletContext,
@Autowired(required = false) UserServiceInterface userService,
TempFileManager tempFileManager,
Environment environment) {
this.servletContext = servletContext;
this.userService = userService;
this.tempFileManager = tempFileManager;
this.environment = environment;
}
/**
* POST to an internal API endpoint. The endpointPath must start with one of the allowed
* prefixes (e.g. {@code /api/v1/misc/compress-pdf}).
*
* @param endpointPath API path (e.g. {@code /api/v1/general/rotate-pdf})
* @param body multipart form body (fileInput + parameters)
* @return response with the result file as a {@link TempFileResource} body
*/
public ResponseEntity<Resource> post(String endpointPath, MultiValueMap<String, Object> body) {
validateUrl(endpointPath);
String url = getBaseUrl() + endpointPath;
RestTemplate restTemplate = new RestTemplate();
HttpHeaders headers = new HttpHeaders();
String apiKey = getApiKeyForUser();
if (apiKey != null && !apiKey.isEmpty()) {
headers.add("X-API-KEY", apiKey);
}
HttpEntity<MultiValueMap<String, Object>> entity = new HttpEntity<>(body, headers);
RequestCallback requestCallback = restTemplate.httpEntityCallback(entity, Resource.class);
return restTemplate.execute(
url,
HttpMethod.POST,
requestCallback,
response -> {
try {
TempFile tempFile = tempFileManager.createManagedTempFile("internal-api");
Files.copy(
response.getBody(),
tempFile.getPath(),
java.nio.file.StandardCopyOption.REPLACE_EXISTING);
String filename = extractFilename(response.getHeaders());
TempFileResource resource = new TempFileResource(tempFile, filename);
return ResponseEntity.status(response.getStatusCode())
.headers(response.getHeaders())
.body(resource);
} catch (IOException e) {
throw new UncheckedIOException(e);
}
});
}
/**
* Extract the filename from a response's {@code Content-Disposition} header. Returns {@code
* null} if the header is missing or has no filename.
*/
private static String extractFilename(HttpHeaders headers) {
String contentDisposition = headers.getFirst(HttpHeaders.CONTENT_DISPOSITION);
if (contentDisposition == null || contentDisposition.isBlank()) {
return null;
}
for (String part : contentDisposition.split(";")) {
String trimmed = part.trim();
if (trimmed.startsWith("filename")) {
String[] kv = trimmed.split("=", 2);
if (kv.length != 2) {
continue;
}
String value = kv[1].trim().replace("\"", "");
return URLDecoder.decode(value, StandardCharsets.UTF_8);
}
}
return null;
}
private String getBaseUrl() {
// Resolve the port lazily so desktop mode (server.port=0, OS-assigned) dispatches to the
// actual bound port. Spring publishes local.server.port once the web server is up; fall
// back to the configured server.port for early calls (tests, non-web contexts).
String port = environment.getProperty("local.server.port");
if (port == null) {
port = environment.getProperty("server.port", "8080");
}
return "http://localhost:" + port + servletContext.getContextPath();
}
private String getApiKeyForUser() {
if (userService == null) return "";
String username = userService.getCurrentUsername();
if (username != null && !username.equals("anonymousUser")) {
return userService.getApiKeyForUser(username);
}
return userService.getApiKeyForUser(Role.INTERNAL_API_USER.getRoleId());
}
private void validateUrl(String endpointPath) {
if (endpointPath == null || !ALLOWED_ENDPOINT_PATH.matcher(endpointPath).matches()) {
log.warn("Blocked internal API request to disallowed path: {}", endpointPath);
throw new SecurityException(
"Internal API dispatch not permitted for endpoint: " + endpointPath);
}
}
/**
* A {@link FileSystemResource} that holds a reference to its backing {@link TempFile}.
*
* <p>If a display filename is supplied (typically parsed from the upstream response's {@code
* Content-Disposition} header), it is returned from {@link #getFilename()} instead of the
* underlying temp file's path-based name.
*/
public static class TempFileResource extends FileSystemResource {
private final TempFile tempFile;
private final String displayFilename;
public TempFileResource(TempFile tempFile) {
this(tempFile, null);
}
public TempFileResource(TempFile tempFile, String displayFilename) {
super(tempFile.getFile());
this.tempFile = tempFile;
this.displayFilename = displayFilename;
}
public TempFile getTempFile() {
return tempFile;
}
@Override
public String getFilename() {
return displayFilename != null ? displayFilename : super.getFilename();
}
}
}
@@ -16,6 +16,7 @@ import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.stereotype.Service;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import jakarta.servlet.http.HttpServletRequest;
@@ -305,33 +306,21 @@ public class JobExecutorService {
Object body = response.getBody();
if (body instanceof byte[]) {
// Extract filename from content-disposition header if available
String filename = "result.pdf";
String contentType = MediaType.APPLICATION_PDF_VALUE;
String filename = extractResponseFilename(response);
String contentType = extractResponseContentType(response);
if (response.getHeaders().getContentDisposition() != null) {
String disposition =
response.getHeaders().getContentDisposition().toString();
if (disposition.contains("filename=")) {
filename =
disposition.substring(
disposition.indexOf("filename=") + 9,
disposition.lastIndexOf('"'));
}
}
MediaType mediaType = response.getHeaders().getContentType();
if (mediaType != null) {
contentType = mediaType.toString();
}
// Store byte array directly to disk
String fileId = fileStorage.storeBytes((byte[]) body, filename);
taskManager.setFileResult(jobId, fileId, filename, contentType);
log.debug("Stored ResponseEntity<byte[]> result with fileId: {}", fileId);
} else if (body instanceof StreamingResponseBody streamingBody) {
String filename = extractResponseFilename(response);
String contentType = extractResponseContentType(response);
// Let the GC handle the memory naturally
String fileId = fileStorage.storeFromStreamingBody(streamingBody, filename);
taskManager.setFileResult(jobId, fileId, filename, contentType);
log.debug(
"Stored ResponseEntity<StreamingResponseBody> result with fileId: {}",
fileId);
} else {
// Check if the response body contains a fileId
if (body != null && body.toString().contains("fileId")) {
@@ -481,6 +470,21 @@ public class JobExecutorService {
}
}
private static String extractResponseFilename(ResponseEntity<?> response) {
if (response.getHeaders().getContentDisposition() != null) {
String filename = response.getHeaders().getContentDisposition().getFilename();
if (filename != null && !filename.isEmpty()) {
return filename;
}
}
return "result.pdf";
}
private static String extractResponseContentType(ResponseEntity<?> response) {
MediaType mediaType = response.getHeaders().getContentType();
return mediaType != null ? mediaType.toString() : MediaType.APPLICATION_PDF_VALUE;
}
/**
* Parse session timeout string (e.g., "30m", "1h") to milliseconds
*
@@ -401,7 +401,7 @@ public class JobQueue implements SmartLifecycle {
* @throws Exception If there is an execution error
*/
private <T> T executeWithTimeout(Supplier<T> supplier, long timeoutMs) throws Exception {
CompletableFuture<T> future = CompletableFuture.supplyAsync(supplier);
CompletableFuture<T> future = CompletableFuture.supplyAsync(supplier, jobExecutor);
try {
if (timeoutMs <= 0) {
@@ -7,11 +7,8 @@ import java.lang.management.MemoryMXBean;
import java.lang.management.OperatingSystemMXBean;
import java.lang.management.RuntimeMXBean;
import java.lang.management.ThreadMXBean;
import java.net.InetAddress;
import java.net.NetworkInterface;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.Enumeration;
import java.util.HashMap;
import java.util.Locale;
import java.util.Map;
@@ -94,21 +91,12 @@ public class PostHogService {
metrics.put("os_name", System.getProperty("os.name"));
metrics.put("os_version", System.getProperty("os.version"));
metrics.put("java_version", System.getProperty("java.version"));
metrics.put("user_name", System.getProperty("user.name"));
metrics.put("user_home", System.getProperty("user.home"));
metrics.put("user_dir", System.getProperty("user.dir"));
// CPU and Memory
metrics.put("cpu_cores", Runtime.getRuntime().availableProcessors());
metrics.put("total_memory", Runtime.getRuntime().totalMemory());
metrics.put("free_memory", Runtime.getRuntime().freeMemory());
// Network and Server Identity
InetAddress localHost = InetAddress.getLocalHost();
metrics.put("ip_address", localHost.getHostAddress());
metrics.put("hostname", localHost.getHostName());
metrics.put("mac_address", getMacAddress());
// JVM info
metrics.put("jvm_vendor", System.getProperty("java.vendor"));
metrics.put("jvm_version", System.getProperty("java.vm.version"));
@@ -153,9 +141,6 @@ public class PostHogService {
metrics.put("gc_" + gcBean.getName() + "_time", gcBean.getCollectionTime());
}
// Network interfaces
metrics.put("network_interfaces", getNetworkInterfacesInfo());
// Docker detection and stats
boolean isDocker = isRunningInDocker();
if (isDocker) {
@@ -353,30 +338,6 @@ public class PostHogService {
.getProFeatures()
.getCustomMetadata()
.isAutoUpdateMetadata());
addIfNotEmpty(
properties,
"enterpriseEdition_customMetadata_author",
applicationProperties
.getPremium()
.getProFeatures()
.getCustomMetadata()
.getAuthor());
addIfNotEmpty(
properties,
"enterpriseEdition_customMetadata_creator",
applicationProperties
.getPremium()
.getProFeatures()
.getCustomMetadata()
.getCreator());
addIfNotEmpty(
properties,
"enterpriseEdition_customMetadata_producer",
applicationProperties
.getPremium()
.getProFeatures()
.getCustomMetadata()
.getProducer());
}
// Capture AutoPipeline properties
addIfNotEmpty(
@@ -386,39 +347,4 @@ public class PostHogService {
return properties;
}
private String getMacAddress() {
try {
Enumeration<NetworkInterface> networkInterfaces =
NetworkInterface.getNetworkInterfaces();
while (networkInterfaces.hasMoreElements()) {
NetworkInterface ni = networkInterfaces.nextElement();
byte[] hardwareAddress = ni.getHardwareAddress();
if (hardwareAddress != null) {
String[] hexadecimal = new String[hardwareAddress.length];
for (int i = 0; i < hardwareAddress.length; i++) {
hexadecimal[i] = String.format("%02X", hardwareAddress[i]);
}
return String.join("-", hexadecimal);
}
}
} catch (Exception e) {
// Handle exception
}
return "Unknown";
}
private Map<String, String> getNetworkInterfacesInfo() {
Map<String, String> interfacesInfo = new HashMap<>();
try {
Enumeration<NetworkInterface> nets = NetworkInterface.getNetworkInterfaces();
while (nets.hasMoreElements()) {
NetworkInterface netint = nets.nextElement();
interfacesInfo.put(netint.getName(), netint.getDisplayName());
}
} catch (Exception e) {
interfacesInfo.put("error", e.getMessage());
}
return interfacesInfo;
}
}
@@ -0,0 +1,18 @@
package stirling.software.common.service;
/** Provides metadata about tool endpoints for internal dispatch. */
public interface ToolMetadataService {
/** Returns true if the given operation path accepts multiple input files. */
boolean isMultiInput(String operationPath);
/**
* Returns true when the endpoint's ZIP response is a transport for multiple typed results and
* should be unpacked: multi-output endpoints (Type:SIMO / Type:MIMO) and wrapper declarations
* such as {@code Output:ZIP-PDF} or {@code Output:IMAGE/ZIP}.
*
* <p>Returns false for a bare {@code Output:ZIP} (e.g. {@code get-attachments}), where the
* archive itself is the deliverable and should be kept packed.
*/
boolean shouldUnpackZipResponse(String operationPath);
}
@@ -5,6 +5,8 @@ public interface UserServiceInterface {
String getCurrentUsername();
String getCurrentUserApiKey();
long getTotalUsersCount();
boolean isCurrentUserAdmin();
@@ -112,8 +112,6 @@ public class FileMonitor {
All files observed changes in the last iteration will be considered as staging files.
If those files are not modified in current iteration, they will be considered as ready for processing.
*/
stagingFiles = new HashSet<>(newlyDiscoveredFiles);
readyForProcessingFiles.clear();
if (path2KeyMapping.isEmpty()) {
log.warn("Not monitoring any directories; attempting to re-register root paths.");
@@ -129,8 +127,19 @@ public class FileMonitor {
}
}
WatchKey key;
while ((key = watchService.poll()) != null) {
// Skip expensive collection work when there is nothing to track
WatchKey firstKey = watchService.poll();
if (firstKey == null
&& newlyDiscoveredFiles.isEmpty()
&& readyForProcessingFiles.isEmpty()) {
return;
}
stagingFiles = new HashSet<>(newlyDiscoveredFiles);
readyForProcessingFiles.clear();
WatchKey key = firstKey;
while (key != null) {
final Path watchingDir = (Path) key.watchable();
key.pollEvents()
.forEach(
@@ -167,6 +176,7 @@ public class FileMonitor {
if (!isKeyValid) { // key is invalid when the directory itself is no longer exists
path2KeyMapping.remove((Path) key.watchable());
}
key = watchService.poll();
}
readyForProcessingFiles.addAll(stagingFiles);
}
@@ -1,6 +1,5 @@
package stirling.software.common.util;
import java.io.ByteArrayInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.UncheckedIOException;
@@ -66,16 +65,7 @@ public class FileToPdf {
ProcessExecutor.getInstance(ProcessExecutor.Processes.WEASYPRINT)
.runCommandWithOutputHandling(command);
byte[] pdfBytes = Files.readAllBytes(tempOutputFile.getPath());
try {
return pdfBytes;
} catch (Exception e) {
pdfBytes = Files.readAllBytes(tempOutputFile.getPath());
if (pdfBytes.length < 1) {
throw e;
}
return pdfBytes;
}
return Files.readAllBytes(tempOutputFile.getPath());
} // tempInputFile auto-closed
} // tempOutputFile auto-closed
}
@@ -92,8 +82,7 @@ public class FileToPdf {
throws IOException {
try (TempDirectory tempUnzippedDir = new TempDirectory(tempFileManager)) {
try (ZipInputStream zipIn =
ZipSecurity.createHardenedInputStream(
new ByteArrayInputStream(Files.readAllBytes(zipFilePath)))) {
ZipSecurity.createHardenedInputStream(Files.newInputStream(zipFilePath))) {
ZipEntry entry = zipIn.getNextEntry();
while (entry != null) {
Path filePath =
@@ -629,11 +629,12 @@ public class FormUtils {
}
log.debug("Skipping form fill because document has no AcroForm");
if (flatten) {
flattenEntireDocument(document, null);
flattenEntireDocument(document, null, false);
}
return;
}
boolean valuesApplied = false;
if (values != null && !values.isEmpty()) {
acroForm.setCacheFields(true);
@@ -667,18 +668,26 @@ public class FormUtils {
Object rawValue = entry.getValue();
String value = rawValue == null ? null : Objects.toString(rawValue, null);
applyValueToField(field, value, strict);
valuesApplied = true;
}
ensureAppearances(acroForm);
if (valuesApplied) {
ensureAppearances(acroForm);
}
}
repairWidgetGeometry(document, acroForm);
if (flatten) {
flattenEntireDocument(document, acroForm);
flattenEntireDocument(document, acroForm, valuesApplied);
}
}
// Cap the fallback rendering DPI. This path only runs when acroForm.flatten()
// throws, and the goal is a readable flattened document — not print quality —
// so clamping avoids runaway memory/CPU on pathological inputs.
private static final int FLATTEN_FALLBACK_MAX_DPI = 200;
private void flattenViaRendering(PDDocument document, PDAcroForm acroForm) throws IOException {
if (document == null) {
return;
@@ -704,28 +713,34 @@ public class FormUtils {
properties != null && properties.getSystem() != null
? properties.getSystem().getMaxDPI()
: 300;
int effectiveDpi = Math.min(requestedDpi, FLATTEN_FALLBACK_MAX_DPI);
rebuildDocumentFromImages(document, renderer, requestedDpi);
rebuildDocumentFromImages(document, renderer, effectiveDpi);
}
// note: this implementation suffers from:
// https://issues.apache.org/jira/browse/PDFBOX-5962
private void flattenEntireDocument(PDDocument document, PDAcroForm acroForm)
throws IOException {
if (document == null) {
// Use PDFBox's built-in field flattening which bakes form field values
// into the page content stream as static text/graphics, removing the
// interactive form structure but preserving all other document content
// (images, text, annotations, etc.) at full quality.
//
// Forcing appearance regeneration via setNeedAppearances(true) drives
// PDFBox into refreshAppearances inside flatten(), where it can hang on
// certain documents (PDFBOX-5962). We therefore only regenerate when we
// actually wrote new values, or when some widgets are missing appearance
// streams and would otherwise flatten blank.
private void flattenEntireDocument(
PDDocument document, PDAcroForm acroForm, boolean valuesWritten) throws IOException {
if (document == null || acroForm == null) {
return;
}
if (acroForm == null) {
return;
}
// Use PDFBox's built-in field flattening which bakes form field values
// into the page content stream as static text/graphics, removing the
// interactive form structure but preserving all other document content
// (images, text, annotations, etc.) at full quality.
try {
if (valuesWritten || hasWidgetWithoutAppearance(acroForm)) {
ensureAppearances(acroForm);
} else {
acroForm.setNeedAppearances(false);
}
try {
acroForm.flatten();
} catch (Exception e) {
log.warn(
@@ -736,6 +751,28 @@ public class FormUtils {
}
}
private boolean hasWidgetWithoutAppearance(PDAcroForm acroForm) {
for (PDField field : acroForm.getFieldTree()) {
if (!(field instanceof PDTerminalField terminalField)) {
continue;
}
List<PDAnnotationWidget> widgets = terminalField.getWidgets();
if (widgets == null) {
continue;
}
for (PDAnnotationWidget widget : widgets) {
if (widget == null) {
continue;
}
PDAppearanceDictionary appearance = widget.getAppearance();
if (appearance == null || appearance.getNormalAppearance() == null) {
return true;
}
}
}
return false;
}
private void rebuildDocumentFromImages(PDDocument document, PDFRenderer renderer, int dpi)
throws IOException {
int pageCount = document.getNumberOfPages();
@@ -1183,4 +1183,25 @@ public class GeneralUtils {
}
}
}
public String getLocalNetworkIp() {
try {
Enumeration<NetworkInterface> interfaces = NetworkInterface.getNetworkInterfaces();
if (interfaces == null) return null;
while (interfaces.hasMoreElements()) {
NetworkInterface iface = interfaces.nextElement();
if (!iface.isUp() || iface.isLoopback() || iface.isVirtual()) continue;
Enumeration<InetAddress> addresses = iface.getInetAddresses();
while (addresses.hasMoreElements()) {
InetAddress addr = addresses.nextElement();
if (addr instanceof Inet4Address && addr.isSiteLocalAddress()) {
return addr.getHostAddress();
}
}
}
} catch (Exception e) {
log.warn("Failed to detect local network IP", e);
}
return null;
}
}
@@ -1,9 +1,9 @@
package stirling.software.common.util;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.ArrayList;
@@ -20,6 +20,7 @@ import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import com.vladsch.flexmark.html2md.converter.FlexmarkHtmlConverter;
import com.vladsch.flexmark.util.data.MutableDataSet;
@@ -48,7 +49,7 @@ public class PDFToFile {
this.runtimePathConfig = runtimePathConfig;
}
public ResponseEntity<byte[]> processPdfToMarkdown(MultipartFile inputFile)
public ResponseEntity<StreamingResponseBody> processPdfToMarkdown(MultipartFile inputFile)
throws IOException, InterruptedException {
if (!MediaType.APPLICATION_PDF_VALUE.equals(inputFile.getContentType())) {
return new ResponseEntity<>(HttpStatus.BAD_REQUEST);
@@ -85,78 +86,77 @@ public class PDFToFile {
pdfBaseName = originalPdfFileName.substring(0, originalPdfFileName.lastIndexOf('.'));
}
byte[] fileBytes;
String fileName;
String fileName = pdfBaseName + "ToMarkdown.zip";
TempFile finalOut = tempFileManager.createManagedTempFile(".zip");
try {
try (TempFile tempInputFile = new TempFile(tempFileManager, ".pdf");
TempDirectory tempOutputDir = new TempDirectory(tempFileManager)) {
inputFile.transferTo(tempInputFile.getFile());
try (TempFile tempInputFile = new TempFile(tempFileManager, ".pdf");
TempDirectory tempOutputDir = new TempDirectory(tempFileManager)) {
inputFile.transferTo(tempInputFile.getFile());
List<String> command =
new ArrayList<>(
Arrays.asList(
"pdftohtml",
"-s",
"-noframes",
"-c",
tempInputFile.getAbsolutePath(),
pdfBaseName));
List<String> command =
new ArrayList<>(
Arrays.asList(
"pdftohtml",
"-s",
"-noframes",
"-c",
tempInputFile.getAbsolutePath(),
pdfBaseName));
ProcessExecutorResult returnCode =
ProcessExecutor.getInstance(ProcessExecutor.Processes.PDFTOHTML)
.runCommandWithOutputHandling(
command, tempOutputDir.getPath().toFile());
// Process HTML files to Markdown
File[] outputFiles =
Objects.requireNonNull(tempOutputDir.getPath().toFile().listFiles());
List<File> markdownFiles = new ArrayList<>();
List<File> imageFiles = new ArrayList<>();
ProcessExecutorResult returnCode =
ProcessExecutor.getInstance(ProcessExecutor.Processes.PDFTOHTML)
.runCommandWithOutputHandling(
command, tempOutputDir.getPath().toFile());
// Process HTML files to Markdown
File[] outputFiles =
Objects.requireNonNull(tempOutputDir.getPath().toFile().listFiles());
List<File> markdownFiles = new ArrayList<>();
List<File> imageFiles = new ArrayList<>();
// Convert HTML files to Markdown and collect image files
for (File outputFile : outputFiles) {
if (outputFile.getName().endsWith(".html")) {
String html = Files.readString(outputFile.toPath());
String markdown = htmlToMarkdownConverter.convert(html);
// Convert HTML files to Markdown and collect image files
for (File outputFile : outputFiles) {
if (outputFile.getName().endsWith(".html")) {
String html = Files.readString(outputFile.toPath());
String markdown = htmlToMarkdownConverter.convert(html);
// Update image references to point to images/ folder
markdown = updateImageReferences(markdown);
// Update image references to point to images/ folder
markdown = updateImageReferences(markdown);
String mdFileName = outputFile.getName().replace(".html", ".md");
File mdFile = new File(tempOutputDir.getPath().toFile(), mdFileName);
Files.writeString(mdFile.toPath(), markdown);
markdownFiles.add(mdFile);
} else if (!outputFile.getName().endsWith(".md")) {
// Collect non-HTML, non-MD files as images/assets
imageFiles.add(outputFile);
}
}
String mdFileName = outputFile.getName().replace(".html", ".md");
File mdFile = new File(tempOutputDir.getPath().toFile(), mdFileName);
Files.writeString(mdFile.toPath(), markdown);
markdownFiles.add(mdFile);
} else if (!outputFile.getName().endsWith(".md")) {
// Collect non-HTML, non-MD files as images/assets
imageFiles.add(outputFile);
try (OutputStream fos = Files.newOutputStream(finalOut.getPath());
ZipOutputStream zipOutputStream = new ZipOutputStream(fos)) {
// Add markdown files to root of ZIP
for (File mdFile : markdownFiles) {
ZipEntry mdEntry = new ZipEntry(mdFile.getName());
zipOutputStream.putNextEntry(mdEntry);
Files.copy(mdFile.toPath(), zipOutputStream);
zipOutputStream.closeEntry();
}
// Add images and other assets to images/ folder
for (File imageFile : imageFiles) {
ZipEntry assetEntry = new ZipEntry("images/" + imageFile.getName());
zipOutputStream.putNextEntry(assetEntry);
Files.copy(imageFile.toPath(), zipOutputStream);
zipOutputStream.closeEntry();
}
}
}
// Always create a ZIP file
fileName = pdfBaseName + "ToMarkdown.zip";
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
try (ZipOutputStream zipOutputStream = new ZipOutputStream(byteArrayOutputStream)) {
// Add markdown files to root of ZIP
for (File mdFile : markdownFiles) {
ZipEntry mdEntry = new ZipEntry(mdFile.getName());
zipOutputStream.putNextEntry(mdEntry);
Files.copy(mdFile.toPath(), zipOutputStream);
zipOutputStream.closeEntry();
}
// Add images and other assets to images/ folder
for (File imageFile : imageFiles) {
ZipEntry assetEntry = new ZipEntry("images/" + imageFile.getName());
zipOutputStream.putNextEntry(assetEntry);
Files.copy(imageFile.toPath(), zipOutputStream);
zipOutputStream.closeEntry();
}
}
fileBytes = byteArrayOutputStream.toByteArray();
} catch (Exception e) {
finalOut.close();
throw e;
}
return WebResponseUtils.bytesToWebResponse(
fileBytes, fileName, MediaType.APPLICATION_OCTET_STREAM);
return WebResponseUtils.fileToWebResponse(
finalOut, fileName, MediaType.APPLICATION_OCTET_STREAM);
}
/**
@@ -169,7 +169,7 @@ public class PDFToFile {
return PATTERN.matcher(markdown).replaceAll("$1(images/$2)");
}
public ResponseEntity<byte[]> processPdfToHtml(MultipartFile inputFile)
public ResponseEntity<StreamingResponseBody> processPdfToHtml(MultipartFile inputFile)
throws IOException, InterruptedException {
if (!MediaType.APPLICATION_PDF_VALUE.equals(inputFile.getContentType())) {
return new ResponseEntity<>(HttpStatus.BAD_REQUEST);
@@ -182,56 +182,57 @@ public class PDFToFile {
pdfBaseName = originalPdfFileName.substring(0, originalPdfFileName.lastIndexOf('.'));
}
byte[] fileBytes;
String fileName;
String fileName = pdfBaseName + "ToHtml.zip";
TempFile finalOut = tempFileManager.createManagedTempFile(".zip");
try {
try (TempFile inputFileTemp = new TempFile(tempFileManager, ".pdf");
TempDirectory outputDirTemp = new TempDirectory(tempFileManager)) {
try (TempFile inputFileTemp = new TempFile(tempFileManager, ".pdf");
TempDirectory outputDirTemp = new TempDirectory(tempFileManager)) {
Path tempInputFile = inputFileTemp.getPath();
Path tempOutputDir = outputDirTemp.getPath();
Path tempInputFile = inputFileTemp.getPath();
Path tempOutputDir = outputDirTemp.getPath();
// Save the uploaded file to a temporary location
inputFile.transferTo(tempInputFile);
// Save the uploaded file to a temporary location
inputFile.transferTo(tempInputFile);
// Run the pdftohtml command with complex output
List<String> command =
new ArrayList<>(
Arrays.asList(
"pdftohtml", "-c", tempInputFile.toString(), pdfBaseName));
// Run the pdftohtml command with complex output
List<String> command =
new ArrayList<>(
Arrays.asList(
"pdftohtml", "-c", tempInputFile.toString(), pdfBaseName));
ProcessExecutorResult returnCode =
ProcessExecutor.getInstance(ProcessExecutor.Processes.PDFTOHTML)
.runCommandWithOutputHandling(command, tempOutputDir.toFile());
ProcessExecutorResult returnCode =
ProcessExecutor.getInstance(ProcessExecutor.Processes.PDFTOHTML)
.runCommandWithOutputHandling(command, tempOutputDir.toFile());
// Get output files
File[] outputFiles = Objects.requireNonNull(tempOutputDir.toFile().listFiles());
// Get output files
File[] outputFiles = Objects.requireNonNull(tempOutputDir.toFile().listFiles());
// Return output files in a ZIP archive
fileName = pdfBaseName + "ToHtml.zip";
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
try (ZipOutputStream zipOutputStream = new ZipOutputStream(byteArrayOutputStream)) {
for (File outputFile : outputFiles) {
ZipEntry entry = new ZipEntry(outputFile.getName());
zipOutputStream.putNextEntry(entry);
try (FileInputStream fis = new FileInputStream(outputFile)) {
IOUtils.copy(fis, zipOutputStream);
} catch (IOException e) {
log.error("Exception writing zip entry", e);
try (OutputStream fos = Files.newOutputStream(finalOut.getPath());
ZipOutputStream zipOutputStream = new ZipOutputStream(fos)) {
for (File outputFile : outputFiles) {
ZipEntry entry = new ZipEntry(outputFile.getName());
zipOutputStream.putNextEntry(entry);
try (FileInputStream fis = new FileInputStream(outputFile)) {
IOUtils.copy(fis, zipOutputStream);
} catch (IOException e) {
log.error("Exception writing zip entry", e);
}
zipOutputStream.closeEntry();
}
zipOutputStream.closeEntry();
} catch (IOException e) {
log.error("Exception writing zip", e);
}
} catch (IOException e) {
log.error("Exception writing zip", e);
}
fileBytes = byteArrayOutputStream.toByteArray();
} catch (Exception e) {
finalOut.close();
throw e;
}
return WebResponseUtils.bytesToWebResponse(
fileBytes, fileName, MediaType.APPLICATION_OCTET_STREAM);
return WebResponseUtils.fileToWebResponse(
finalOut, fileName, MediaType.APPLICATION_OCTET_STREAM);
}
public ResponseEntity<byte[]> processPdfToOfficeFormat(
public ResponseEntity<StreamingResponseBody> processPdfToOfficeFormat(
MultipartFile inputFile, String outputFormat, String libreOfficeFilter)
throws IOException, InterruptedException {
@@ -257,109 +258,115 @@ public class PDFToFile {
return new ResponseEntity<>(HttpStatus.BAD_REQUEST);
}
byte[] fileBytes;
String fileName;
TempFile finalOut =
tempFileManager.createManagedTempFile("." + resolvePrimaryExtension(outputFormat));
Path libreOfficeProfile = null;
try (TempFile inputFileTemp = new TempFile(tempFileManager, ".pdf");
TempDirectory outputDirTemp = new TempDirectory(tempFileManager)) {
try {
try (TempFile inputFileTemp = new TempFile(tempFileManager, ".pdf");
TempDirectory outputDirTemp = new TempDirectory(tempFileManager)) {
Path tempInputFile = inputFileTemp.getPath();
Path tempOutputDir = outputDirTemp.getPath();
Path unoOutputFile =
tempOutputDir.resolve(
pdfBaseName + "." + resolvePrimaryExtension(outputFormat));
Path tempInputFile = inputFileTemp.getPath();
Path tempOutputDir = outputDirTemp.getPath();
Path unoOutputFile =
tempOutputDir.resolve(
pdfBaseName + "." + resolvePrimaryExtension(outputFormat));
// Save the uploaded file to a temporary location
inputFile.transferTo(tempInputFile);
// Save the uploaded file to a temporary location
inputFile.transferTo(tempInputFile);
// Run the LibreOffice command
ProcessExecutorResult returnCode = null;
IOException unoconvertException = null;
// Run the LibreOffice command
ProcessExecutorResult returnCode = null;
IOException unoconvertException = null;
if (isUnoConvertEnabled()) {
try {
List<String> unoCommand =
buildUnoConvertCommand(
tempInputFile, unoOutputFile, outputFormat, libreOfficeFilter);
returnCode =
ProcessExecutor.getInstance(ProcessExecutor.Processes.LIBRE_OFFICE)
.runCommandWithOutputHandling(unoCommand);
} catch (IOException e) {
unoconvertException = e;
log.warn(
"Unoconvert command failed ({}). Falling back to soffice command.",
e.getMessage());
}
}
if (returnCode == null) {
// Run the LibreOffice command as a fallback
libreOfficeProfile = Files.createTempDirectory("libreoffice_profile_");
List<String> command = new ArrayList<>();
command.add(runtimePathConfig.getSOfficePath());
command.add("-env:UserInstallation=" + libreOfficeProfile.toUri().toString());
command.add("--headless");
command.add("--nologo");
command.add("--infilter=" + libreOfficeFilter);
command.add("--convert-to");
command.add(outputFormat);
command.add("--outdir");
command.add(tempOutputDir.toString());
command.add(tempInputFile.toString());
try {
returnCode =
ProcessExecutor.getInstance(ProcessExecutor.Processes.LIBRE_OFFICE)
.runCommandWithOutputHandling(command);
} catch (IOException e) {
if (unoconvertException != null) {
e.addSuppressed(unoconvertException);
if (isUnoConvertEnabled()) {
try {
List<String> unoCommand =
buildUnoConvertCommand(
tempInputFile,
unoOutputFile,
outputFormat,
libreOfficeFilter);
returnCode =
ProcessExecutor.getInstance(ProcessExecutor.Processes.LIBRE_OFFICE)
.runCommandWithOutputHandling(unoCommand);
} catch (IOException e) {
unoconvertException = e;
log.warn(
"Unoconvert command failed ({}). Falling back to soffice command.",
e.getMessage());
}
throw e;
}
}
// Get output files
List<File> outputFiles = Arrays.asList(tempOutputDir.toFile().listFiles());
if (returnCode == null) {
// Run the LibreOffice command as a fallback
libreOfficeProfile = Files.createTempDirectory("libreoffice_profile_");
List<String> command = new ArrayList<>();
command.add(runtimePathConfig.getSOfficePath());
command.add("-env:UserInstallation=" + libreOfficeProfile.toUri().toString());
command.add("--headless");
command.add("--nologo");
command.add("--infilter=" + libreOfficeFilter);
command.add("--convert-to");
command.add(outputFormat);
command.add("--outdir");
command.add(tempOutputDir.toString());
command.add(tempInputFile.toString());
if (outputFiles.size() == 1) {
// Return single output file
File outputFile = outputFiles.get(0);
if ("txt:Text".equals(outputFormat)) {
outputFormat = "txt";
}
fileName = pdfBaseName + "." + outputFormat;
fileBytes = FileUtils.readFileToByteArray(outputFile);
} else {
// Return output files in a ZIP archive
fileName = pdfBaseName + "To" + outputFormat + ".zip";
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
try (ZipOutputStream zipOutputStream = new ZipOutputStream(byteArrayOutputStream)) {
for (File outputFile : outputFiles) {
ZipEntry entry = new ZipEntry(outputFile.getName());
zipOutputStream.putNextEntry(entry);
try (FileInputStream fis = new FileInputStream(outputFile)) {
IOUtils.copy(fis, zipOutputStream);
} catch (IOException e) {
log.error("Exception writing zip entry", e);
try {
returnCode =
ProcessExecutor.getInstance(ProcessExecutor.Processes.LIBRE_OFFICE)
.runCommandWithOutputHandling(command);
} catch (IOException e) {
if (unoconvertException != null) {
e.addSuppressed(unoconvertException);
}
zipOutputStream.closeEntry();
throw e;
}
} catch (IOException e) {
log.error("Exception writing zip", e);
}
fileBytes = byteArrayOutputStream.toByteArray();
// Get output files
List<File> outputFiles = Arrays.asList(tempOutputDir.toFile().listFiles());
if (outputFiles.size() == 1) {
// Return single output file
File outputFile = outputFiles.get(0);
if ("txt:Text".equals(outputFormat)) {
outputFormat = "txt";
}
fileName = pdfBaseName + "." + outputFormat;
FileUtils.copyFile(outputFile, finalOut.getFile());
} else {
// Return output files in a ZIP archive
fileName = pdfBaseName + "To" + outputFormat + ".zip";
try (OutputStream fos = Files.newOutputStream(finalOut.getPath());
ZipOutputStream zipOutputStream = new ZipOutputStream(fos)) {
for (File outputFile : outputFiles) {
ZipEntry entry = new ZipEntry(outputFile.getName());
zipOutputStream.putNextEntry(entry);
try (FileInputStream fis = new FileInputStream(outputFile)) {
IOUtils.copy(fis, zipOutputStream);
} catch (IOException e) {
log.error("Exception writing zip entry", e);
}
zipOutputStream.closeEntry();
}
} catch (IOException e) {
log.error("Exception writing zip", e);
}
}
}
} catch (Exception e) {
finalOut.close();
throw e;
} finally {
if (libreOfficeProfile != null) {
FileUtils.deleteQuietly(libreOfficeProfile.toFile());
}
}
return WebResponseUtils.bytesToWebResponse(
fileBytes, fileName, MediaType.APPLICATION_OCTET_STREAM);
return WebResponseUtils.fileToWebResponse(
finalOut, fileName, MediaType.APPLICATION_OCTET_STREAM);
}
private boolean isUnoConvertEnabled() {
@@ -282,8 +282,9 @@ public class ProcessExecutor {
boolean finished = process.waitFor(timeoutDuration, TimeUnit.MINUTES);
if (!finished) {
// Terminate the process
process.destroy();
// Kill the entire process tree (descendants first, then the process itself)
process.descendants().forEach(ProcessHandle::destroyForcibly);
process.destroyForcibly();
// Interrupt the reader threads
errorReaderThread.interrupt();
outputReaderThread.interrupt();
@@ -538,9 +538,9 @@ public final class RegexPatternUtils {
getPattern("[^a-zA-Z0-9 ]"); // Input sanitization
getPattern("[^a-zA-Z0-9]"); // Filename sanitization
// API doc patterns
getPattern("Output:(\\w+)"); // precompiled single-escaped for runtime regex \w
getPattern("Input:(\\w+)");
getPattern("Type:(\\w+)");
getPattern("Output:\\s*(\\w+)");
getPattern("Input:\\s*(\\w+)");
getPattern("Type:\\s*(\\w+)");
log.debug("Pre-compiled {} common regex patterns", patternCache.size());
}
@@ -552,19 +552,19 @@ public final class RegexPatternUtils {
/* Pattern for matching Output:<TYPE> in API descriptions */
public Pattern getApiDocOutputTypePattern() {
return getPattern("Output:(\\w+)");
return getPattern("Output:\\s*(\\w+)");
}
/* Pattern for matching Input:<TYPE> in API descriptions */
public Pattern getApiDocInputTypePattern() {
return getPattern("Input:(\\w+)");
return getPattern("Input:\\s*(\\w+)");
}
/**
* Pattern for matching Type:<CODE> in API descriptions
*/
public Pattern getApiDocTypePattern() {
return getPattern("Type:(\\w+)");
return getPattern("Type:\\s*(\\w+)");
}
/* Pattern for validating file extensions (2-4 alphanumeric, case-insensitive) */
@@ -73,6 +73,19 @@ public class WebResponseUtils {
return baosToWebResponse(baos, docName);
}
public static ResponseEntity<StreamingResponseBody> pdfDocToWebResponse(
PDDocument document, String docName, TempFileManager tempFileManager)
throws IOException {
TempFile tempFile = tempFileManager.createManagedTempFile(".pdf");
try {
document.save(tempFile.getFile());
} catch (IOException e) {
tempFile.close();
throw e;
}
return pdfFileToWebResponse(tempFile, docName);
}
/**
* Convert a File to a web response (PDF default).
*
@@ -108,23 +121,37 @@ public class WebResponseUtils {
public static ResponseEntity<StreamingResponseBody> fileToWebResponse(
TempFile outputTempFile, String docName, MediaType mediaType) throws IOException {
Path path = outputTempFile.getFile().toPath().normalize();
long len = Files.size(path);
HttpHeaders headers = new HttpHeaders();
headers.setContentType(mediaType);
headers.setContentLength(len);
headers.add(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=\"" + docName + "\"");
try {
Path path = outputTempFile.getFile().toPath().normalize();
long len = Files.size(path);
HttpHeaders headers = new HttpHeaders();
headers.setContentType(mediaType);
headers.setContentLength(len);
String encodedDocName =
RegexPatternUtils.getInstance()
.getPlusSignPattern()
.matcher(URLEncoder.encode(docName, StandardCharsets.UTF_8))
.replaceAll("%20");
headers.setContentDispositionFormData("attachment", encodedDocName);
StreamingResponseBody body =
os -> {
try (os) {
Files.copy(path, os);
os.flush();
} finally {
outputTempFile.close();
}
};
StreamingResponseBody body =
os -> {
try (os) {
Files.copy(path, os);
os.flush();
} finally {
outputTempFile.close();
}
};
return new ResponseEntity<>(body, headers, HttpStatus.OK);
return new ResponseEntity<>(body, headers, HttpStatus.OK);
} catch (IOException | RuntimeException e) {
try {
outputTempFile.close();
} catch (Exception closeEx) {
e.addSuppressed(closeEx);
}
throw e;
}
}
}
@@ -0,0 +1,142 @@
package stirling.software.common.util;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.nio.file.Files;
import java.util.ArrayList;
import java.util.List;
import java.util.function.Consumer;
import java.util.zip.ZipEntry;
import java.util.zip.ZipInputStream;
import org.springframework.core.io.FileSystemResource;
import org.springframework.core.io.Resource;
import io.github.pixee.security.ZipSecurity;
import lombok.experimental.UtilityClass;
import lombok.extern.slf4j.Slf4j;
/**
* Helpers for detecting and extracting ZIP-formatted responses returned from Stirling API
* endpoints. Shared between {@code PipelineProcessor} and {@code AiWorkflowService} so both callers
* unpack ZIPs consistently (hardened against zip-slip, depth-limited, backed by managed temp
* files).
*/
@Slf4j
@UtilityClass
public class ZipExtractionUtils {
private static final int MAX_UNZIP_DEPTH = 10;
private static final byte[] ZIP_MAGIC = {0x50, 0x4B, 0x03, 0x04};
/**
* Returns true if the resource starts with the standard ZIP magic bytes. CBZ files are
* explicitly treated as non-ZIP.
*/
public static boolean isZip(Resource data) throws IOException {
return isZip(data, null);
}
/**
* Returns true if the resource starts with the standard ZIP magic bytes. Files named with the
* {@code .cbz} extension are excluded (handled separately by the comic viewer).
*/
public static boolean isZip(Resource data, String filename) throws IOException {
if (data == null || data.contentLength() < ZIP_MAGIC.length) {
return false;
}
if (filename != null && filename.toLowerCase().endsWith(".cbz")) {
return false;
}
try (InputStream is = data.getInputStream()) {
byte[] header = new byte[ZIP_MAGIC.length];
if (is.read(header) < ZIP_MAGIC.length) {
return false;
}
for (int i = 0; i < ZIP_MAGIC.length; i++) {
if (header[i] != ZIP_MAGIC[i]) {
return false;
}
}
return true;
}
}
/**
* Extract a ZIP resource into a flat list of resources, one per file entry. Nested ZIPs are
* recursively extracted up to {@link #MAX_UNZIP_DEPTH}. Each entry is materialized as a
* hardened-extracted managed temp file so downstream consumers can stream the bytes.
*/
public static List<Resource> extractZip(Resource zip, TempFileManager tempFileManager)
throws IOException {
return extractZip(zip, tempFileManager, null);
}
/**
* Extract a ZIP resource into a flat list of resources. Each created {@link TempFile} is also
* passed to {@code tempFileConsumer} when non-null, giving callers the option to register the
* temp files with an auxiliary lifecycle (e.g. {@code PipelineResult}).
*/
public static List<Resource> extractZip(
Resource zip, TempFileManager tempFileManager, Consumer<TempFile> tempFileConsumer)
throws IOException {
return extractZipInternal(zip, tempFileManager, tempFileConsumer, 0);
}
private static List<Resource> extractZipInternal(
Resource zip,
TempFileManager tempFileManager,
Consumer<TempFile> tempFileConsumer,
int depth)
throws IOException {
if (depth > MAX_UNZIP_DEPTH) {
log.warn(
"ZIP nesting depth {} exceeds limit {}, treating as file",
depth,
MAX_UNZIP_DEPTH);
return List.of(zip);
}
log.debug("Unzipping data of length: {}", zip.contentLength());
List<Resource> extracted = new ArrayList<>();
try (InputStream bais = zip.getInputStream();
ZipInputStream zis = ZipSecurity.createHardenedInputStream(bais)) {
ZipEntry entry;
while ((entry = zis.getNextEntry()) != null) {
if (entry.isDirectory()) {
continue;
}
TempFile tempFile = tempFileManager.createManagedTempFile("unzip");
if (tempFileConsumer != null) {
tempFileConsumer.accept(tempFile);
}
try (OutputStream os = Files.newOutputStream(tempFile.getPath())) {
byte[] buffer = new byte[4096];
int count;
while ((count = zis.read(buffer)) != -1) {
os.write(buffer, 0, count);
}
}
final String filename = entry.getName();
Resource fileResource =
new FileSystemResource(tempFile.getFile()) {
@Override
public String getFilename() {
return filename;
}
};
if (isZip(fileResource, filename)) {
log.debug("Nested ZIP entry {} — recursing", filename);
extracted.addAll(
extractZipInternal(
fileResource, tempFileManager, tempFileConsumer, depth + 1));
} else {
extracted.add(fileResource);
}
}
}
log.debug("Unzipping completed. {} files extracted.", extracted.size());
return extracted;
}
}
@@ -0,0 +1,159 @@
package stirling.software.common.service;
import static org.junit.jupiter.api.Assertions.*;
import static org.mockito.ArgumentMatchers.*;
import static org.mockito.Mockito.*;
import java.io.ByteArrayInputStream;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import org.springframework.core.io.ByteArrayResource;
import org.springframework.core.io.Resource;
import org.springframework.http.*;
import org.springframework.http.client.ClientHttpResponse;
import org.springframework.mock.env.MockEnvironment;
import org.springframework.util.LinkedMultiValueMap;
import org.springframework.util.MultiValueMap;
import org.springframework.web.client.RequestCallback;
import org.springframework.web.client.ResponseExtractor;
import org.springframework.web.client.RestTemplate;
import jakarta.servlet.ServletContext;
import stirling.software.common.util.TempFile;
import stirling.software.common.util.TempFileManager;
@ExtendWith(MockitoExtension.class)
class InternalApiClientTest {
@Mock ServletContext servletContext;
@Mock UserServiceInterface userService;
@Mock TempFileManager tempFileManager;
InternalApiClient client;
@BeforeEach
void setUp() {
lenient().when(servletContext.getContextPath()).thenReturn("");
MockEnvironment environment = new MockEnvironment().withProperty("server.port", "8080");
client = new InternalApiClient(servletContext, userService, tempFileManager, environment);
}
@Test
void postDoesNotForceContentType() throws Exception {
MultiValueMap<String, Object> body = new LinkedMultiValueMap<>();
body.add("fileInput", namedResource("input.pdf", "data"));
Path tempPath = Files.createTempFile("internal-api-test", ".tmp");
TempFile tempFile = mock(TempFile.class);
when(tempFile.getPath()).thenReturn(tempPath);
when(tempFile.getFile()).thenReturn(tempPath.toFile());
when(tempFileManager.createManagedTempFile("internal-api")).thenReturn(tempFile);
HttpHeaders[] captured = {null};
try (var ignored =
mockConstruction(
RestTemplate.class,
(rt, ctx) -> {
when(rt.httpEntityCallback(any(), eq(Resource.class)))
.thenAnswer(
inv -> {
HttpEntity<?> entity = inv.getArgument(0);
captured[0] = entity.getHeaders();
return (RequestCallback) req -> {};
});
when(rt.execute(anyString(), eq(HttpMethod.POST), any(), any()))
.thenAnswer(inv -> fakeOkResponse(inv.getArgument(3)));
})) {
ResponseEntity<Resource> response = client.post("/api/v1/general/merge-pdfs", body);
assertNotNull(response);
assertEquals(HttpStatus.OK, response.getStatusCode());
assertNotNull(response.getBody());
assertNull(captured[0].getContentType(), "Content-Type should not be forced");
} finally {
Files.deleteIfExists(tempPath);
}
}
@Test
void postRejectsDisallowedPath() {
MultiValueMap<String, Object> body = new LinkedMultiValueMap<>();
assertThrows(SecurityException.class, () -> client.post("/api/v1/admin/settings", body));
}
@Test
void postRejectsPathTraversal() {
MultiValueMap<String, Object> body = new LinkedMultiValueMap<>();
assertThrows(
SecurityException.class,
() -> client.post("/api/v1/misc/../../actuator/env", body));
}
@Test
void postRejectsUrlEncodedCharacters() {
MultiValueMap<String, Object> body = new LinkedMultiValueMap<>();
assertThrows(
SecurityException.class, () -> client.post("/api/v1/misc/%2e%2e/actuator", body));
}
@Test
void postRejectsQueryString() {
MultiValueMap<String, Object> body = new LinkedMultiValueMap<>();
assertThrows(
SecurityException.class,
() -> client.post("/api/v1/misc/compress-pdf?redirect=evil", body));
}
@Test
void postRejectsEmptySegment() {
MultiValueMap<String, Object> body = new LinkedMultiValueMap<>();
assertThrows(SecurityException.class, () -> client.post("/api/v1/misc//foo", body));
}
@Test
void postRejectsTrailingSlash() {
MultiValueMap<String, Object> body = new LinkedMultiValueMap<>();
assertThrows(SecurityException.class, () -> client.post("/api/v1/misc/foo/", body));
}
@Test
void postRejectsNullPath() {
MultiValueMap<String, Object> body = new LinkedMultiValueMap<>();
assertThrows(SecurityException.class, () -> client.post(null, body));
}
/** Create a ByteArrayResource with a filename (required for multipart). */
private static Resource namedResource(String filename, String content) {
return new ByteArrayResource(content.getBytes(StandardCharsets.UTF_8)) {
@Override
public String getFilename() {
return filename;
}
};
}
/** Simulate a successful HTTP response through a RestTemplate ResponseExtractor. */
@SuppressWarnings("unchecked")
private static ResponseEntity<Resource> fakeOkResponse(Object extractorArg) throws Exception {
var extractor = (ResponseExtractor<ResponseEntity<Resource>>) extractorArg;
ClientHttpResponse response = mock(ClientHttpResponse.class);
when(response.getBody())
.thenReturn(new ByteArrayInputStream("ok".getBytes(StandardCharsets.UTF_8)));
HttpHeaders headers = new HttpHeaders();
headers.add(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=\"out.pdf\"");
when(response.getHeaders()).thenReturn(headers);
lenient().when(response.getStatusCode()).thenReturn(HttpStatus.OK);
return extractor.extractData(response);
}
}
@@ -3,6 +3,7 @@ package stirling.software.common.util;
import static org.junit.jupiter.api.Assertions.*;
import java.io.IOException;
import java.time.Duration;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
@@ -249,6 +250,60 @@ class FormUtilsAdditionalTest {
}
}
// Regression: PDFBOX-5962. Flattening with an empty values map used to force
// setNeedAppearances(true), triggering PDFBox's refreshAppearances loop which
// could hang indefinitely. The call must complete quickly and clear form fields.
@Test
void testApplyFieldValues_emptyValuesWithFlatten_completesAndFlattens() throws IOException {
try (PDDocument doc = new PDDocument()) {
SetupDocument setup = createBasicDocument(doc);
PDTextField textField = new PDTextField(setup.acroForm);
textField.setPartialName("company");
attachWidget(setup, textField, new PDRectangle(60, 720, 220, 20));
assertTrue(setup.acroForm.getNeedAppearances());
assertTimeoutPreemptively(
Duration.ofSeconds(10),
() -> FormUtils.applyFieldValues(doc, Map.of(), true, false));
PDAcroForm after = doc.getDocumentCatalog().getAcroForm();
assertTrue(after == null || after.getFields().isEmpty());
}
}
@Test
void testApplyFieldValues_nullValuesWithFlatten_completesAndFlattens() throws IOException {
try (PDDocument doc = new PDDocument()) {
SetupDocument setup = createBasicDocument(doc);
PDTextField textField = new PDTextField(setup.acroForm);
textField.setPartialName("company");
attachWidget(setup, textField, new PDRectangle(60, 720, 220, 20));
assertTimeoutPreemptively(
Duration.ofSeconds(10),
() -> FormUtils.applyFieldValues(doc, null, true, false));
PDAcroForm after = doc.getDocumentCatalog().getAcroForm();
assertTrue(after == null || after.getFields().isEmpty());
}
}
@Test
void testApplyFieldValues_valuesWithFlatten_appliesValueAndFlattens() throws IOException {
try (PDDocument doc = new PDDocument()) {
SetupDocument setup = createBasicDocument(doc);
PDTextField textField = new PDTextField(setup.acroForm);
textField.setPartialName("company");
attachWidget(setup, textField, new PDRectangle(60, 720, 220, 20));
FormUtils.applyFieldValues(doc, Map.of("company", "Stirling"), true, false);
PDAcroForm after = doc.getDocumentCatalog().getAcroForm();
assertTrue(after == null || after.getFields().isEmpty());
}
}
// --- filterSingleChoiceSelection ---
@Test
@@ -9,6 +9,7 @@ import static org.mockito.Mockito.lenient;
import static org.mockito.Mockito.mockStatic;
import static org.mockito.Mockito.when;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.IOException;
import java.nio.file.Files;
@@ -29,6 +30,7 @@ import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.mock.web.MockMultipartFile;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import io.github.pixee.security.ZipSecurity;
@@ -59,6 +61,19 @@ class PDFToFileTest {
.thenAnswer(
invocation ->
Files.createTempFile("test", invocation.getArgument(0)).toFile());
lenient()
.when(mockTempFileManager.createManagedTempFile(anyString()))
.thenAnswer(
invocation -> {
File f =
Files.createTempFile("test", invocation.<String>getArgument(0))
.toFile();
TempFile tf = org.mockito.Mockito.mock(TempFile.class);
lenient().when(tf.getFile()).thenReturn(f);
lenient().when(tf.getPath()).thenReturn(f.toPath());
lenient().when(tf.getAbsolutePath()).thenReturn(f.getAbsolutePath());
return tf;
});
lenient()
.when(mockTempFileManager.createTempDirectory())
.thenAnswer(invocation -> Files.createTempDirectory("test"));
@@ -68,6 +83,12 @@ class PDFToFileTest {
pdfToFile = new PDFToFile(mockTempFileManager, mockRuntimePathConfig);
}
private static byte[] drain(ResponseEntity<StreamingResponseBody> response) throws IOException {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
response.getBody().writeTo(baos);
return baos.toByteArray();
}
@Test
void testProcessPdfToMarkdown_InvalidContentType() throws IOException, InterruptedException {
// Prepare
@@ -79,7 +100,7 @@ class PDFToFileTest {
"This is not a PDF".getBytes());
// Execute
ResponseEntity<byte[]> response = pdfToFile.processPdfToMarkdown(nonPdfFile);
ResponseEntity<StreamingResponseBody> response = pdfToFile.processPdfToMarkdown(nonPdfFile);
// Verify
assertEquals(HttpStatus.BAD_REQUEST, response.getStatusCode());
@@ -96,7 +117,7 @@ class PDFToFileTest {
"This is not a PDF".getBytes());
// Execute
ResponseEntity<byte[]> response = pdfToFile.processPdfToHtml(nonPdfFile);
ResponseEntity<StreamingResponseBody> response = pdfToFile.processPdfToHtml(nonPdfFile);
// Verify
assertEquals(HttpStatus.BAD_REQUEST, response.getStatusCode());
@@ -114,7 +135,7 @@ class PDFToFileTest {
"This is not a PDF".getBytes());
// Execute
ResponseEntity<byte[]> response =
ResponseEntity<StreamingResponseBody> response =
pdfToFile.processPdfToOfficeFormat(nonPdfFile, "docx", "draw_pdf_import");
// Verify
@@ -133,7 +154,7 @@ class PDFToFileTest {
"Fake PDF content".getBytes());
// Execute with invalid format
ResponseEntity<byte[]> response =
ResponseEntity<StreamingResponseBody> response =
pdfToFile.processPdfToOfficeFormat(pdfFile, "invalid_format", "draw_pdf_import");
// Verify
@@ -184,12 +205,14 @@ class PDFToFileTest {
});
// Execute the method
ResponseEntity<byte[]> response = pdfToFile.processPdfToMarkdown(pdfFile);
ResponseEntity<StreamingResponseBody> response =
pdfToFile.processPdfToMarkdown(pdfFile);
// Verify - should now return a ZIP file instead of plain markdown
assertEquals(HttpStatus.OK, response.getStatusCode());
assertNotNull(response.getBody());
assertTrue(response.getBody().length > 0);
byte[] bodyBytes = drain(response);
assertNotNull(bodyBytes);
assertTrue(bodyBytes.length > 0);
// Verify content disposition indicates a ZIP file
assertTrue(
@@ -201,7 +224,7 @@ class PDFToFileTest {
// Verify the content by unzipping it
try (ZipInputStream zipStream =
ZipSecurity.createHardenedInputStream(
new java.io.ByteArrayInputStream(response.getBody()))) {
new java.io.ByteArrayInputStream(bodyBytes))) {
ZipEntry entry;
boolean foundMdFile = false;
boolean foundImageInFolder = false;
@@ -275,12 +298,14 @@ class PDFToFileTest {
});
// Execute the method
ResponseEntity<byte[]> response = pdfToFile.processPdfToMarkdown(pdfFile);
ResponseEntity<StreamingResponseBody> response =
pdfToFile.processPdfToMarkdown(pdfFile);
// Verify
assertEquals(HttpStatus.OK, response.getStatusCode());
assertNotNull(response.getBody());
assertTrue(response.getBody().length > 0);
byte[] bodyBytes = drain(response);
assertNotNull(bodyBytes);
assertTrue(bodyBytes.length > 0);
// Verify content disposition indicates a zip file
assertTrue(
@@ -292,7 +317,7 @@ class PDFToFileTest {
// Verify the content by unzipping it
try (ZipInputStream zipStream =
ZipSecurity.createHardenedInputStream(
new java.io.ByteArrayInputStream(response.getBody()))) {
new java.io.ByteArrayInputStream(bodyBytes))) {
ZipEntry entry;
boolean foundMdFiles = false;
boolean foundImage = false;
@@ -352,12 +377,13 @@ class PDFToFileTest {
});
// Execute the method
ResponseEntity<byte[]> response = pdfToFile.processPdfToHtml(pdfFile);
ResponseEntity<StreamingResponseBody> response = pdfToFile.processPdfToHtml(pdfFile);
// Verify
assertEquals(HttpStatus.OK, response.getStatusCode());
assertNotNull(response.getBody());
assertTrue(response.getBody().length > 0);
byte[] bodyBytes = drain(response);
assertNotNull(bodyBytes);
assertTrue(bodyBytes.length > 0);
// Verify content disposition indicates a zip file
assertTrue(
@@ -369,7 +395,7 @@ class PDFToFileTest {
// Verify the content by unzipping it
try (ZipInputStream zipStream =
ZipSecurity.createHardenedInputStream(
new java.io.ByteArrayInputStream(response.getBody()))) {
new java.io.ByteArrayInputStream(bodyBytes))) {
ZipEntry entry;
boolean foundMainHtml = false;
boolean foundIndexHtml = false;
@@ -437,13 +463,14 @@ class PDFToFileTest {
});
// Execute the method with docx format
ResponseEntity<byte[]> response =
ResponseEntity<StreamingResponseBody> response =
pdfToFile.processPdfToOfficeFormat(pdfFile, "docx", "draw_pdf_import");
// Verify
assertEquals(HttpStatus.OK, response.getStatusCode());
assertNotNull(response.getBody());
assertTrue(response.getBody().length > 0);
byte[] bodyBytes = drain(response);
assertNotNull(bodyBytes);
assertTrue(bodyBytes.length > 0);
// Verify content disposition has correct filename
assertTrue(
@@ -508,13 +535,14 @@ class PDFToFileTest {
});
// Execute the method with ODP format
ResponseEntity<byte[]> response =
ResponseEntity<StreamingResponseBody> response =
pdfToFile.processPdfToOfficeFormat(pdfFile, "odp", "draw_pdf_import");
// Verify
assertEquals(HttpStatus.OK, response.getStatusCode());
assertNotNull(response.getBody());
assertTrue(response.getBody().length > 0);
byte[] bodyBytes = drain(response);
assertNotNull(bodyBytes);
assertTrue(bodyBytes.length > 0);
// Verify content disposition for zip file
assertTrue(
@@ -526,7 +554,7 @@ class PDFToFileTest {
// Verify the content by unzipping it
try (ZipInputStream zipStream =
ZipSecurity.createHardenedInputStream(
new java.io.ByteArrayInputStream(response.getBody()))) {
new java.io.ByteArrayInputStream(bodyBytes))) {
ZipEntry entry;
boolean foundMainFile = false;
boolean foundMediaFiles = false;
@@ -592,13 +620,14 @@ class PDFToFileTest {
});
// Execute the method with text format
ResponseEntity<byte[]> response =
ResponseEntity<StreamingResponseBody> response =
pdfToFile.processPdfToOfficeFormat(pdfFile, "txt:Text", "draw_pdf_import");
// Verify
assertEquals(HttpStatus.OK, response.getStatusCode());
assertNotNull(response.getBody());
assertTrue(response.getBody().length > 0);
byte[] bodyBytes = drain(response);
assertNotNull(bodyBytes);
assertTrue(bodyBytes.length > 0);
// Verify content disposition has txt extension
assertTrue(
@@ -650,13 +679,14 @@ class PDFToFileTest {
});
// Execute the method
ResponseEntity<byte[]> response =
ResponseEntity<StreamingResponseBody> response =
pdfToFile.processPdfToOfficeFormat(pdfFile, "docx", "draw_pdf_import");
// Verify
assertEquals(HttpStatus.OK, response.getStatusCode());
assertNotNull(response.getBody());
assertTrue(response.getBody().length > 0);
byte[] bodyBytes = drain(response);
assertNotNull(bodyBytes);
assertTrue(bodyBytes.length > 0);
// Verify content disposition contains output.docx
assertTrue(
@@ -696,12 +726,13 @@ class PDFToFileTest {
return mockExecutorResult;
});
ResponseEntity<byte[]> response =
ResponseEntity<StreamingResponseBody> response =
pdfToFileWithUno.processPdfToOfficeFormat(pdfFile, "docx", "writer_pdf_import");
assertEquals(HttpStatus.OK, response.getStatusCode());
assertNotNull(response.getBody());
assertTrue(response.getBody().length > 0);
byte[] bodyBytes = drain(response);
assertNotNull(bodyBytes);
assertTrue(bodyBytes.length > 0);
assertTrue(
response.getHeaders()
.getContentDisposition()
@@ -759,12 +790,13 @@ class PDFToFileTest {
return mockExecutorResult;
});
ResponseEntity<byte[]> response =
ResponseEntity<StreamingResponseBody> response =
pdfToFileWithUno.processPdfToOfficeFormat(pdfFile, "docx", "writer_pdf_import");
assertEquals(HttpStatus.OK, response.getStatusCode());
assertNotNull(response.getBody());
assertTrue(response.getBody().length > 0);
byte[] bodyBytes = drain(response);
assertNotNull(bodyBytes);
assertTrue(bodyBytes.length > 0);
assertTrue(
response.getHeaders()
.getContentDisposition()
+7 -5
View File
@@ -14,6 +14,8 @@ spotless {
target 'src/**/java/**/*.java'
targetExclude 'src/main/resources/static/**', 'src/main/java/org/apache/**'
googleJavaFormat(googleJavaFormatVersion).aosp().reorderImports(false)
// google-java-format 1.28.0 bundles Guava 32.x which crashes Spotless lint on JDK 24/25
suppressLintsFor { setStep('google-java-format') }
importOrder("java", "javax", "org", "com", "net", "io", "jakarta", "lombok", "me", "stirling")
trimTrailingWhitespace()
@@ -66,7 +68,7 @@ dependencies {
implementation 'commons-io:commons-io:2.21.0'
implementation "org.bouncycastle:bcprov-jdk18on:$bouncycastleVersion"
implementation "org.bouncycastle:bcpkix-jdk18on:$bouncycastleVersion"
implementation 'io.micrometer:micrometer-core:1.16.2'
implementation 'io.micrometer:micrometer-core'
implementation 'com.google.zxing:core:3.5.4'
implementation "org.commonmark:commonmark:$commonmarkVersion" // https://mvnrepository.com/artifact/org.commonmark/commonmark
implementation "org.commonmark:commonmark-ext-gfm-tables:$commonmarkVersion"
@@ -82,7 +84,7 @@ dependencies {
// veraPDF still uses javax.xml.bind, not the new jakarta namespace
implementation 'javax.xml.bind:jaxb-api:2.3.1'
implementation 'com.sun.xml.bind:jaxb-impl:2.3.9'
implementation 'com.sun.xml.bind:jaxb-core:4.0.6'
implementation 'com.sun.xml.bind:jaxb-core:4.0.7'
implementation 'org.apache.poi:poi-ooxml:5.5.1'
// https://mvnrepository.com/artifact/technology.tabula/tabula
@@ -176,6 +178,7 @@ springBoot {
// Frontend build tasks - only enabled with -PbuildWithFrontend=true
def buildWithFrontend = project.hasProperty('buildWithFrontend') && project.property('buildWithFrontend') == 'true'
def buildPrototypes = project.hasProperty('prototypesMode') && project.property('prototypesMode') == 'true'
def frontendDir = file('../../frontend')
def frontendDistDir = file('../../frontend/dist')
def resourcesStaticDir = file('src/main/resources/static')
@@ -243,9 +246,8 @@ tasks.register('npmBuild', Exec) {
enabled = buildWithFrontend
group = 'frontend'
description = 'Build frontend application'
workingDir frontendDir
commandLine = Os.isFamily(Os.FAMILY_WINDOWS) ? ['cmd', '/c', 'npm', 'run', 'build'] : ['npm', 'run', 'build']
dependsOn npmInstall
workingDir file('../..')
commandLine = buildPrototypes ? ['task', 'frontend:build:prototypes'] : ['task', 'frontend:build']
inputs.dir(new File(frontendDir, 'src'))
inputs.dir(new File(frontendDir, 'public'))
inputs.file(new File(frontendDir, 'package.json'))
@@ -73,7 +73,8 @@ public class ExternalAppDepConfig {
tmp.put("tesseract", List.of("tesseract"));
tmp.put("rar", List.of("rar")); // Required for real CBR output
tmp.put(calibrePath, List.of("Calibre"));
tmp.put("ffmpeg", List.of("FFmpeg"));
// ffmpeg disabled due to raised CVEs
// tmp.put("ffmpeg", List.of("FFmpeg"));
tmp.put("magick", List.of("ImageMagick"));
this.commandToGroupMapping = Collections.unmodifiableMap(tmp);
}
@@ -47,7 +47,7 @@ public class OpenApiConfig {
.version(version)
.license(
new License()
.name("MIT")
.name("Open-Core - MIT Licensed")
.url(
"https://raw.githubusercontent.com/Stirling-Tools/Stirling-PDF/refs/heads/main/LICENSE"))
.termsOfService("https://www.stirlingpdf.com/terms")
@@ -28,7 +28,17 @@ public class SpringDocConfig {
"/api/v1/proprietary/ui-data/**",
"/api/v1/info/**",
"/api/v1/general/job/**",
"/api/v1/general/files/**")
"/api/v1/general/files/**",
"/api/v1/general/signatures/**",
"/api/v1/database/**",
"/api/v1/storage/**",
"/api/v1/proprietary/signatures/**",
"/api/v1/workflow/participant/**",
"/api/v1/security/cert-sign/sessions",
"/api/v1/security/cert-sign/sessions/**",
"/api/v1/security/cert-sign/sign-requests",
"/api/v1/security/cert-sign/sign-requests/**",
"/api/v1/security/cert-sign/validate-certificate")
.addOpenApiCustomizer(pdfFileOneOfCustomizer)
.addOpenApiCustomizer(
openApi -> {
@@ -53,7 +63,16 @@ public class SpringDocConfig {
"/api/v1/team/**",
"/api/v1/auth/**",
"/api/v1/invite/**",
"/api/v1/audit/**")
"/api/v1/audit/**",
"/api/v1/database/**",
"/api/v1/storage/**",
"/api/v1/proprietary/signatures/**",
"/api/v1/workflow/participant/**",
"/api/v1/security/cert-sign/sessions",
"/api/v1/security/cert-sign/sessions/**",
"/api/v1/security/cert-sign/sign-requests",
"/api/v1/security/cert-sign/sign-requests/**",
"/api/v1/security/cert-sign/validate-certificate")
.addOpenApiCustomizer(
openApi -> {
openApi.info(
@@ -75,7 +94,8 @@ public class SpringDocConfig {
"/api/v1/proprietary/ui-data/**",
"/api/v1/info/**",
"/api/v1/general/job/**",
"/api/v1/general/files/**")
"/api/v1/general/files/**",
"/api/v1/general/signatures/**")
.addOpenApiCustomizer(
openApi -> {
openApi.info(
@@ -1,7 +1,6 @@
package stirling.software.SPDF.config;
import java.lang.management.ManagementFactory;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;
@@ -106,25 +105,27 @@ public class TauriProcessMonitor {
logger.info("Orphaned Java backend detected. Shutting down gracefully...");
// Shutdown asynchronously to avoid blocking the monitor thread
CompletableFuture.runAsync(
() -> {
try {
// Give a small delay to ensure logging completes
Thread.sleep(1000);
Thread.ofVirtual()
.name("tauri-graceful-shutdown")
.start(
() -> {
try {
// Give a small delay to ensure logging completes
Thread.sleep(1000);
if (applicationContext instanceof ConfigurableApplicationContext) {
((ConfigurableApplicationContext) applicationContext).close();
} else {
// Fallback to system exit
logger.warn(
"Unable to shutdown Spring context gracefully, using System.exit");
System.exit(0);
}
} catch (Exception e) {
logger.error("Error during graceful shutdown", e);
System.exit(1);
}
});
if (applicationContext instanceof ConfigurableApplicationContext) {
((ConfigurableApplicationContext) applicationContext).close();
} else {
// Fallback to system exit
logger.warn(
"Unable to shutdown Spring context gracefully, using System.exit");
System.exit(0);
}
} catch (Exception e) {
logger.error("Error during graceful shutdown", e);
System.exit(1);
}
});
}
@PreDestroy
@@ -1,7 +1,6 @@
package stirling.software.SPDF.controller.api;
import java.awt.*;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
@@ -19,6 +18,7 @@ import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import io.github.pixee.security.Filenames;
import io.swagger.v3.oas.annotations.Operation;
@@ -30,6 +30,7 @@ import stirling.software.SPDF.model.api.general.BookletImpositionRequest;
import stirling.software.common.annotations.AutoJobPostMapping;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
@RestController
@@ -39,6 +40,7 @@ import stirling.software.common.util.WebResponseUtils;
public class BookletImpositionController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final TempFileManager tempFileManager;
@AutoJobPostMapping(
value = "/booklet-imposition",
@@ -49,7 +51,7 @@ public class BookletImpositionController {
"This operation combines page reordering for booklet printing with multi-page layout. "
+ "It rearranges pages in the correct order for booklet printing and places multiple pages "
+ "on each sheet for proper folding and binding. Input:PDF Output:PDF Type:SISO")
public ResponseEntity<byte[]> createBookletImposition(
public ResponseEntity<StreamingResponseBody> createBookletImposition(
@ModelAttribute BookletImpositionRequest request) throws IOException {
MultipartFile file = request.getFileInput();
@@ -85,15 +87,12 @@ public class BookletImpositionController {
duplexPass,
flipOnShortEdge)) {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
newDocument.save(baos);
byte[] result = baos.toByteArray();
return WebResponseUtils.bytesToWebResponse(
result,
return WebResponseUtils.pdfDocToWebResponse(
newDocument,
GeneralUtils.generateFilename(
Filenames.toSimpleFileName(file.getOriginalFilename()),
"_booklet.pdf"));
"_booklet.pdf"),
tempFileManager);
}
}
}
@@ -1,10 +1,7 @@
package stirling.software.SPDF.controller.api;
import java.awt.image.BufferedImage;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.List;
import org.apache.pdfbox.multipdf.LayerUtility;
@@ -18,6 +15,7 @@ import org.apache.pdfbox.rendering.PDFRenderer;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import io.swagger.v3.oas.annotations.Operation;
@@ -32,6 +30,8 @@ import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.ProcessExecutor;
import stirling.software.common.util.TempFile;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
@GeneralApi
@@ -46,6 +46,7 @@ public class CropController {
private static final String PDF_EXTENSION = ".pdf";
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final TempFileManager tempFileManager;
private static int[] detectContentBounds(BufferedImage image) {
int width = image.getWidth();
@@ -131,7 +132,8 @@ public class CropController {
description =
"This operation takes an input PDF file and crops it according to the given"
+ " coordinates. Input:PDF Output:PDF Type:SISO")
public ResponseEntity<byte[]> cropPdf(@ModelAttribute CropPdfForm request) throws IOException {
public ResponseEntity<StreamingResponseBody> cropPdf(@ModelAttribute CropPdfForm request)
throws IOException {
if (request.isAutoCrop()) {
return cropWithAutomaticDetection(request);
}
@@ -151,8 +153,8 @@ public class CropController {
}
}
private ResponseEntity<byte[]> cropWithAutomaticDetection(@ModelAttribute CropPdfForm request)
throws IOException {
private ResponseEntity<StreamingResponseBody> cropWithAutomaticDetection(
@ModelAttribute CropPdfForm request) throws IOException {
try (PDDocument sourceDocument = pdfDocumentFactory.load(request)) {
try (PDDocument newDocument =
@@ -196,20 +198,17 @@ public class CropController {
cropBounds.height));
}
ByteArrayOutputStream baos = new ByteArrayOutputStream();
newDocument.save(baos);
byte[] pdfContent = baos.toByteArray();
return WebResponseUtils.bytesToWebResponse(
pdfContent,
return WebResponseUtils.pdfDocToWebResponse(
newDocument,
GeneralUtils.generateFilename(
request.getFileInput().getOriginalFilename(), "_cropped.pdf"));
request.getFileInput().getOriginalFilename(), "_cropped.pdf"),
tempFileManager);
}
}
}
private ResponseEntity<byte[]> cropWithPDFBox(@ModelAttribute CropPdfForm request)
throws IOException {
private ResponseEntity<StreamingResponseBody> cropWithPDFBox(
@ModelAttribute CropPdfForm request) throws IOException {
try (PDDocument sourceDocument = pdfDocumentFactory.load(request)) {
try (PDDocument newDocument =
@@ -255,22 +254,19 @@ public class CropController {
request.getHeight()));
}
ByteArrayOutputStream baos = new ByteArrayOutputStream();
newDocument.save(baos);
byte[] pdfContent = baos.toByteArray();
return WebResponseUtils.bytesToWebResponse(
pdfContent,
return WebResponseUtils.pdfDocToWebResponse(
newDocument,
GeneralUtils.generateFilename(
request.getFileInput().getOriginalFilename(), "_cropped.pdf"));
request.getFileInput().getOriginalFilename(), "_cropped.pdf"),
tempFileManager);
}
}
}
private ResponseEntity<byte[]> cropWithGhostscript(@ModelAttribute CropPdfForm request)
throws IOException {
Path tempInputFile = null;
Path tempOutputFile = null;
private ResponseEntity<StreamingResponseBody> cropWithGhostscript(
@ModelAttribute CropPdfForm request) throws IOException {
TempFile tempInputFile = null;
TempFile tempOutputFile = null;
try (PDDocument sourceDocument = pdfDocumentFactory.load(request)) {
for (int i = 0; i < sourceDocument.getNumberOfPages(); i++) {
@@ -284,11 +280,11 @@ public class CropController {
page.setCropBox(cropBox);
}
tempInputFile = Files.createTempFile(TEMP_INPUT_PREFIX, PDF_EXTENSION);
tempOutputFile = Files.createTempFile(TEMP_OUTPUT_PREFIX, PDF_EXTENSION);
tempInputFile = tempFileManager.createManagedTempFile(PDF_EXTENSION);
tempOutputFile = tempFileManager.createManagedTempFile(PDF_EXTENSION);
// Save the source document with crop boxes
sourceDocument.save(tempInputFile.toFile());
sourceDocument.save(tempInputFile.getFile());
// Execute Ghostscript to process the crop boxes
ProcessExecutor processExecutor =
@@ -299,15 +295,15 @@ public class CropController {
"-sDEVICE=pdfwrite",
"-dUseCropBox",
"-o",
tempOutputFile.toString(),
tempInputFile.toString());
tempOutputFile.getAbsolutePath(),
tempInputFile.getAbsolutePath());
processExecutor.runCommandWithOutputHandling(command);
byte[] pdfContent = Files.readAllBytes(tempOutputFile);
return WebResponseUtils.bytesToWebResponse(
pdfContent,
TempFile out = tempOutputFile;
tempOutputFile = null; // ownership transferred to StreamingResponseBody
return WebResponseUtils.pdfFileToWebResponse(
out,
GeneralUtils.generateFilename(
request.getFileInput().getOriginalFilename(), "_cropped.pdf"));
@@ -316,10 +312,10 @@ public class CropController {
throw ExceptionUtils.createProcessingInterruptedException("Ghostscript", e);
} finally {
if (tempInputFile != null) {
Files.deleteIfExists(tempInputFile);
tempInputFile.close();
}
if (tempOutputFile != null) {
Files.deleteIfExists(tempOutputFile);
tempOutputFile.close();
}
}
}
@@ -1,6 +1,5 @@
package stirling.software.SPDF.controller.api;
import java.io.ByteArrayOutputStream;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
@@ -14,6 +13,7 @@ import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.*;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import io.swagger.v3.oas.annotations.Operation;
@@ -27,6 +27,7 @@ import stirling.software.common.annotations.AutoJobPostMapping;
import stirling.software.common.annotations.api.GeneralApi;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
import tools.jackson.core.type.TypeReference;
@@ -39,6 +40,7 @@ public class EditTableOfContentsController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final ObjectMapper objectMapper;
private final TempFileManager tempFileManager;
@AutoJobPostMapping(
value = "/extract-bookmarks",
@@ -149,12 +151,11 @@ public class EditTableOfContentsController {
@Operation(
summary = "Edit Table of Contents",
description = "Add or edit bookmarks/table of contents in a PDF document.")
public ResponseEntity<byte[]> editTableOfContents(
public ResponseEntity<StreamingResponseBody> editTableOfContents(
@ModelAttribute EditTableOfContentsRequest request) throws Exception {
MultipartFile file = request.getFileInput();
try (PDDocument document = pdfDocumentFactory.load(file);
ByteArrayOutputStream baos = new ByteArrayOutputStream()) {
try (PDDocument document = pdfDocumentFactory.load(file)) {
// Parse the bookmark data from JSON
List<BookmarkItem> bookmarks =
@@ -168,13 +169,10 @@ public class EditTableOfContentsController {
// Add bookmarks to the outline
addBookmarksToOutline(document, outline, bookmarks);
// Save the document to a byte array
document.save(baos);
return WebResponseUtils.bytesToWebResponse(
baos.toByteArray(),
return WebResponseUtils.pdfDocToWebResponse(
document,
GeneralUtils.generateFilename(file.getOriginalFilename(), "_with_toc.pdf"),
MediaType.APPLICATION_PDF);
tempFileManager);
}
}
@@ -3,7 +3,6 @@ package stirling.software.SPDF.controller.api;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.nio.file.Files;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Comparator;
@@ -29,6 +28,7 @@ import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import io.swagger.v3.oas.annotations.Operation;
@@ -279,7 +279,7 @@ public class MergeController {
"This endpoint merges multiple PDF files into a single PDF file. The merged"
+ " file will contain all pages from the input files in the order they were"
+ " provided. Input:PDF Output:PDF Type:MISO")
public ResponseEntity<byte[]> mergePdfs(
public ResponseEntity<StreamingResponseBody> mergePdfs(
@ModelAttribute MergePdfsRequest request,
@RequestParam(value = "fileOrder", required = false) String fileOrder)
throws IOException {
@@ -399,12 +399,6 @@ public class MergeController {
String mergedFileName =
GeneralUtils.generateFilename(firstFilename, "_merged_unsigned.pdf");
byte[] pdfBytes;
try {
pdfBytes = Files.readAllBytes(outputTempFile.getPath());
} finally {
outputTempFile.close();
}
return WebResponseUtils.bytesToWebResponse(pdfBytes, mergedFileName);
return WebResponseUtils.pdfFileToWebResponse(outputTempFile, mergedFileName);
}
}
@@ -1,7 +1,6 @@
package stirling.software.SPDF.controller.api;
import java.awt.Color;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import org.apache.pdfbox.multipdf.LayerUtility;
@@ -15,6 +14,7 @@ import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import io.swagger.v3.oas.annotations.Operation;
@@ -28,6 +28,7 @@ import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.GeneralFormCopyUtils;
import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
@GeneralApi
@@ -36,6 +37,7 @@ import stirling.software.common.util.WebResponseUtils;
public class MultiPageLayoutController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final TempFileManager tempFileManager;
@AutoJobPostMapping(
value = "/multi-page-layout",
@@ -45,7 +47,7 @@ public class MultiPageLayoutController {
description =
"This operation takes an input PDF file and the number of pages to merge into a"
+ " single sheet in the output PDF file. Input:PDF Output:PDF Type:SISO")
public ResponseEntity<byte[]> mergeMultiplePagesIntoOne(
public ResponseEntity<StreamingResponseBody> mergeMultiplePagesIntoOne(
@ModelAttribute MergeMultiplePagesRequest request) throws IOException {
int MAX_PAGES = 100000;
@@ -338,13 +340,11 @@ public class MultiPageLayoutController {
}
}
ByteArrayOutputStream baos = new ByteArrayOutputStream();
newDocument.save(baos);
byte[] result = baos.toByteArray();
return WebResponseUtils.bytesToWebResponse(
result,
return WebResponseUtils.pdfDocToWebResponse(
newDocument,
GeneralUtils.generateFilename(
file.getOriginalFilename(), "_multi_page_layout.pdf"));
file.getOriginalFilename(), "_multi_page_layout.pdf"),
tempFileManager);
} // newDocument is closed here
} // sourceDocument is closed here
}
@@ -1,6 +1,5 @@
package stirling.software.SPDF.controller.api;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.IOException;
import java.nio.file.Files;
@@ -16,6 +15,7 @@ import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import io.swagger.v3.oas.annotations.Operation;
@@ -28,6 +28,8 @@ import stirling.software.common.annotations.api.GeneralApi;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.TempFile;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
@GeneralApi
@@ -35,6 +37,7 @@ import stirling.software.common.util.WebResponseUtils;
public class PdfOverlayController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final TempFileManager tempFileManager;
@AutoJobPostMapping(value = "/overlay-pdfs", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
@StandardPdfResponse
@@ -43,8 +46,8 @@ public class PdfOverlayController {
description =
"Overlay PDF files onto a base PDF with different modes: Sequential,"
+ " Interleaved, or Fixed Repeat. Input:PDF Output:PDF Type:MIMO")
public ResponseEntity<byte[]> overlayPdfs(@ModelAttribute OverlayPdfsRequest request)
throws IOException {
public ResponseEntity<StreamingResponseBody> overlayPdfs(
@ModelAttribute OverlayPdfsRequest request) throws IOException {
MultipartFile baseFile = request.getFileInput();
int overlayPos = request.getOverlayPosition();
@@ -52,6 +55,7 @@ public class PdfOverlayController {
File[] overlayPdfFiles = new File[overlayFiles.length];
List<File> tempFiles = new ArrayList<>(); // List to keep track of temporary files
TempFile tempOut = null;
try {
for (int i = 0; i < overlayFiles.length; i++) {
overlayPdfFiles[i] = GeneralUtils.multipartToFile(overlayFiles[i]);
@@ -62,8 +66,7 @@ public class PdfOverlayController {
int[] counts = request.getCounts(); // Used for FixedRepeatOverlay mode
try (PDDocument basePdf = pdfDocumentFactory.load(baseFile);
Overlay overlay = new Overlay();
ByteArrayOutputStream outputStream = new ByteArrayOutputStream()) {
Overlay overlay = new Overlay()) {
Map<Integer, String> overlayGuide =
prepareOverlayGuide(
basePdf.getNumberOfPages(),
@@ -79,15 +82,21 @@ public class PdfOverlayController {
overlay.setOverlayPosition(Overlay.Position.BACKGROUND);
}
overlay.overlay(overlayGuide).save(outputStream);
byte[] data = outputStream.toByteArray();
tempOut = tempFileManager.createManagedTempFile(".pdf");
overlay.overlay(overlayGuide).save(tempOut.getFile());
String outputFilename =
GeneralUtils.generateFilename(
baseFile.getOriginalFilename(), "_overlayed.pdf");
return WebResponseUtils.bytesToWebResponse(
data, outputFilename, MediaType.APPLICATION_PDF);
TempFile out = tempOut;
tempOut = null; // ownership transferred to StreamingResponseBody
return WebResponseUtils.pdfFileToWebResponse(out, outputFilename);
}
} catch (Exception e) {
if (tempOut != null) {
tempOut.close();
}
throw e;
} finally {
for (File overlayPdfFile : overlayPdfFiles) {
if (overlayPdfFile != null) {
@@ -12,6 +12,7 @@ import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import io.swagger.v3.oas.annotations.Operation;
@@ -27,6 +28,7 @@ import stirling.software.common.annotations.api.GeneralApi;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
@GeneralApi
@@ -35,6 +37,7 @@ import stirling.software.common.util.WebResponseUtils;
public class RearrangePagesPDFController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final TempFileManager tempFileManager;
@AutoJobPostMapping(consumes = MediaType.MULTIPART_FORM_DATA_VALUE, value = "/remove-pages")
@StandardPdfResponse
@@ -44,8 +47,8 @@ public class RearrangePagesPDFController {
"This endpoint removes specified pages from a given PDF file. Users can provide"
+ " a comma-separated list of page numbers or ranges to delete. Input:PDF"
+ " Output:PDF Type:SISO")
public ResponseEntity<byte[]> deletePages(@ModelAttribute PDFWithPageNums request)
throws IOException {
public ResponseEntity<StreamingResponseBody> deletePages(
@ModelAttribute PDFWithPageNums request) throws IOException {
MultipartFile pdfFile = request.getFileInput();
String pagesToDelete = request.getPageNumbers();
@@ -67,7 +70,8 @@ public class RearrangePagesPDFController {
return WebResponseUtils.pdfDocToWebResponse(
document,
GeneralUtils.generateFilename(
pdfFile.getOriginalFilename(), "_removed_pages.pdf"));
pdfFile.getOriginalFilename(), "_removed_pages.pdf"),
tempFileManager);
}
}
@@ -224,8 +228,8 @@ public class RearrangePagesPDFController {
+ " order or custom mode. Users can provide a page order as a"
+ " comma-separated list of page numbers or page ranges, or a custom mode."
+ " Input:PDF Output:PDF")
public ResponseEntity<byte[]> rearrangePages(@ModelAttribute RearrangePagesRequest request)
throws IOException {
public ResponseEntity<StreamingResponseBody> rearrangePages(
@ModelAttribute RearrangePagesRequest request) throws IOException {
MultipartFile pdfFile = request.getFileInput();
String pageOrder = request.getPageNumbers();
String sortType = request.getCustomMode();
@@ -264,7 +268,8 @@ public class RearrangePagesPDFController {
return WebResponseUtils.pdfDocToWebResponse(
rearrangedDocument,
GeneralUtils.generateFilename(
pdfFile.getOriginalFilename(), "_rearranged.pdf"));
pdfFile.getOriginalFilename(), "_rearranged.pdf"),
tempFileManager);
}
}
} catch (IOException e) {
@@ -9,6 +9,7 @@ import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import io.swagger.v3.oas.annotations.Operation;
@@ -21,6 +22,7 @@ import stirling.software.common.annotations.api.GeneralApi;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
@GeneralApi
@@ -28,6 +30,7 @@ import stirling.software.common.util.WebResponseUtils;
public class RotationController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final TempFileManager tempFileManager;
@AutoJobPostMapping(consumes = MediaType.MULTIPART_FORM_DATA_VALUE, value = "/rotate-pdf")
@StandardPdfResponse
@@ -36,7 +39,7 @@ public class RotationController {
description =
"This endpoint rotates a given PDF file by a specified angle. The angle must be"
+ " a multiple of 90. Input:PDF Output:PDF Type:SISO")
public ResponseEntity<byte[]> rotatePDF(@ModelAttribute RotatePDFRequest request)
public ResponseEntity<StreamingResponseBody> rotatePDF(@ModelAttribute RotatePDFRequest request)
throws IOException {
MultipartFile pdfFile = request.getFileInput();
Integer angle = request.getAngle();
@@ -60,7 +63,8 @@ public class RotationController {
// Return the rotated PDF as a response
return WebResponseUtils.pdfDocToWebResponse(
document,
GeneralUtils.generateFilename(pdfFile.getOriginalFilename(), "_rotated.pdf"));
GeneralUtils.generateFilename(pdfFile.getOriginalFilename(), "_rotated.pdf"),
tempFileManager);
}
}
}
@@ -1,6 +1,5 @@
package stirling.software.SPDF.controller.api;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
@@ -16,6 +15,7 @@ import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import io.swagger.v3.oas.annotations.Operation;
@@ -28,6 +28,7 @@ import stirling.software.common.annotations.api.GeneralApi;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
@GeneralApi
@@ -36,6 +37,7 @@ import stirling.software.common.util.WebResponseUtils;
public class ScalePagesController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final TempFileManager tempFileManager;
private static PDRectangle getTargetSize(String targetPDRectangle, PDDocument sourceDocument) {
if ("KEEP".equals(targetPDRectangle)) {
@@ -118,16 +120,15 @@ public class ScalePagesController {
description =
"This operation takes an input PDF file and the size to scale the pages to in"
+ " the output PDF file. Input:PDF Output:PDF Type:SISO")
public ResponseEntity<byte[]> scalePages(@ModelAttribute ScalePagesRequest request)
throws IOException {
public ResponseEntity<StreamingResponseBody> scalePages(
@ModelAttribute ScalePagesRequest request) throws IOException {
MultipartFile file = request.getFileInput();
String targetPDRectangle = request.getPageSize();
float scaleFactor = request.getScaleFactor();
try (PDDocument sourceDocument = pdfDocumentFactory.load(file);
PDDocument outputDocument =
pdfDocumentFactory.createNewDocumentBasedOnOldDocument(sourceDocument);
ByteArrayOutputStream baos = new ByteArrayOutputStream()) {
pdfDocumentFactory.createNewDocumentBasedOnOldDocument(sourceDocument)) {
PDRectangle targetSize = getTargetSize(targetPDRectangle, sourceDocument);
@@ -168,11 +169,10 @@ public class ScalePagesController {
}
}
outputDocument.save(baos);
return WebResponseUtils.bytesToWebResponse(
baos.toByteArray(),
GeneralUtils.generateFilename(file.getOriginalFilename(), "_scaled.pdf"));
return WebResponseUtils.pdfDocToWebResponse(
outputDocument,
GeneralUtils.generateFilename(file.getOriginalFilename(), "_scaled.pdf"),
tempFileManager);
}
}
}
@@ -124,7 +124,9 @@ public class SplitPdfByChaptersController {
@MultiFileResponse
@Operation(
summary = "Split PDFs by Chapters",
description = "Splits a PDF into chapters and returns a ZIP file.")
description =
"Splits a PDF into chapters and returns a ZIP file. Input:PDF Output:ZIP-PDF"
+ " Type:SISO")
public ResponseEntity<StreamingResponseBody> splitPdf(
@ModelAttribute SplitPdfByChaptersRequest request) throws Exception {
MultipartFile file = request.getFileInput();
@@ -1,6 +1,5 @@
package stirling.software.SPDF.controller.api;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.nio.file.Files;
import java.util.*;
@@ -20,6 +19,7 @@ import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import io.swagger.v3.oas.annotations.Operation;
@@ -59,8 +59,8 @@ public class SplitPdfBySectionsController {
+ " which page to split, and how to split"
+ " ( halves, thirds, quarters, etc.), both vertically and horizontally."
+ " Input:PDF Output:ZIP-PDF Type:SISO")
public ResponseEntity<byte[]> splitPdf(@Valid @ModelAttribute SplitPdfBySectionsRequest request)
throws Exception {
public ResponseEntity<StreamingResponseBody> splitPdf(
@Valid @ModelAttribute SplitPdfBySectionsRequest request) throws Exception {
MultipartFile file = request.getFileInput();
String pageNumbers = request.getPageNumbers();
SplitTypes splitMode =
@@ -80,9 +80,7 @@ public class SplitPdfBySectionsController {
if (merge) {
try (PDDocument mergedDoc =
pdfDocumentFactory.createNewDocumentBasedOnOldDocument(
sourceDocument);
ByteArrayOutputStream baos = new ByteArrayOutputStream()) {
pdfDocumentFactory.createNewDocumentBasedOnOldDocument(sourceDocument)) {
LayerUtility layerUtility = new LayerUtility(mergedDoc);
for (int pageIndex = 0;
pageIndex < sourceDocument.getNumberOfPages();
@@ -99,11 +97,12 @@ public class SplitPdfBySectionsController {
addPageToTarget(sourceDocument, pageIndex, mergedDoc, layerUtility);
}
}
mergedDoc.save(baos);
return WebResponseUtils.baosToWebResponse(baos, filename + ".pdf");
return WebResponseUtils.pdfDocToWebResponse(
mergedDoc, filename + ".pdf", tempFileManager);
}
} else {
try (TempFile zipTempFile = new TempFile(tempFileManager, ".zip")) {
TempFile zipTempFile = tempFileManager.createManagedTempFile(".zip");
try {
try (ZipOutputStream zipOut =
new ZipOutputStream(Files.newOutputStream(zipTempFile.getPath()))) {
for (int pageIndex = 0;
@@ -161,9 +160,10 @@ public class SplitPdfBySectionsController {
log.error("Error creating ZIP file with split PDF sections", e);
throw e;
}
byte[] zipBytes = Files.readAllBytes(zipTempFile.getPath());
return WebResponseUtils.bytesToWebResponse(
zipBytes, filename + ".zip", MediaType.APPLICATION_OCTET_STREAM);
return WebResponseUtils.zipFileToWebResponse(zipTempFile, filename + ".zip");
} catch (Exception ex) {
zipTempFile.close();
throw ex;
}
}
} catch (Exception e) {
@@ -1,7 +1,6 @@
package stirling.software.SPDF.controller.api;
import java.awt.geom.AffineTransform;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import org.apache.pdfbox.multipdf.LayerUtility;
@@ -12,6 +11,7 @@ import org.apache.pdfbox.pdmodel.graphics.form.PDFormXObject;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import io.swagger.v3.oas.annotations.Operation;
@@ -23,6 +23,7 @@ import stirling.software.common.annotations.api.GeneralApi;
import stirling.software.common.model.api.PDFFile;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
@GeneralApi
@@ -30,6 +31,7 @@ import stirling.software.common.util.WebResponseUtils;
public class ToSinglePageController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final TempFileManager tempFileManager;
@AutoJobPostMapping(
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
@@ -42,7 +44,7 @@ public class ToSinglePageController {
+ " document. The width of the single page will be same as the input's"
+ " width, but the height will be the sum of all the pages' heights."
+ " Input:PDF Output:PDF Type:SISO")
public ResponseEntity<byte[]> pdfToSinglePage(@ModelAttribute PDFFile request)
public ResponseEntity<StreamingResponseBody> pdfToSinglePage(@ModelAttribute PDFFile request)
throws IOException {
// Load the source document
@@ -85,14 +87,11 @@ public class ToSinglePageController {
pageIndex++;
}
ByteArrayOutputStream baos = new ByteArrayOutputStream();
newDocument.save(baos);
byte[] result = baos.toByteArray();
return WebResponseUtils.bytesToWebResponse(
result,
return WebResponseUtils.pdfDocToWebResponse(
newDocument,
GeneralUtils.generateFilename(
request.getFileInput().getOriginalFilename(), "_singlePage.pdf"));
request.getFileInput().getOriginalFilename(), "_singlePage.pdf"),
tempFileManager);
}
}
}
@@ -95,9 +95,8 @@ public class UIDataController {
Resource resource = new ClassPathResource("static/3rdPartyLicenses.json");
try (InputStream is = resource.getInputStream()) {
String json = new String(is.readAllBytes(), StandardCharsets.UTF_8);
Map<String, List<Dependency>> licenseData =
objectMapper.readValue(json, new TypeReference<>() {});
objectMapper.readValue(is, new TypeReference<>() {});
data.setDependencies(licenseData.get("dependencies"));
} catch (IOException e) {
log.error("Failed to load licenses data", e);
@@ -16,6 +16,7 @@ import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import io.github.pixee.security.Filenames;
import io.swagger.v3.oas.annotations.Operation;
@@ -31,6 +32,7 @@ import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.ProcessExecutor;
import stirling.software.common.util.ProcessExecutor.ProcessExecutorResult;
import stirling.software.common.util.TempFile;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
@@ -60,7 +62,7 @@ public class ConvertEbookToPDFController {
description =
"This endpoint converts common eBook formats (EPUB, MOBI, AZW3, FB2, TXT, DOCX)"
+ " to PDF using Calibre. Input:BOOK Output:PDF Type:SISO")
public ResponseEntity<byte[]> convertEbookToPdf(
public ResponseEntity<StreamingResponseBody> convertEbookToPdf(
@ModelAttribute ConvertEbookToPdfRequest request) throws Exception {
if (!isCalibreEnabled()) {
throw new IllegalStateException("Calibre support is disabled");
@@ -140,24 +142,35 @@ public class ConvertEbookToPDFController {
String outputFilename =
GeneralUtils.generateFilename(originalFilename, "_convertedToPDF.pdf");
TempFile tempOut = null;
try {
tempOut = tempFileManager.createManagedTempFile(".pdf");
if (optimizeForEbook) {
byte[] pdfBytes = Files.readAllBytes(outputPath);
try {
byte[] optimizedPdf = GeneralUtils.optimizePdfWithGhostscript(pdfBytes);
return WebResponseUtils.bytesToWebResponse(optimizedPdf, outputFilename);
Files.write(tempOut.getPath(), optimizedPdf);
} catch (IOException e) {
log.warn(
"Ghostscript optimization failed for ebook conversion, returning"
+ " original PDF",
e);
return WebResponseUtils.bytesToWebResponse(pdfBytes, outputFilename);
Files.write(tempOut.getPath(), pdfBytes);
}
} else {
try (PDDocument document = pdfDocumentFactory.load(outputPath.toFile())) {
document.save(tempOut.getFile());
}
}
try (PDDocument document = pdfDocumentFactory.load(outputPath.toFile())) {
return WebResponseUtils.pdfDocToWebResponse(document, outputFilename);
ResponseEntity<StreamingResponseBody> response =
WebResponseUtils.pdfFileToWebResponse(tempOut, outputFilename);
tempOut = null;
return response;
} catch (Exception e) {
if (tempOut != null) {
tempOut.close();
}
throw e;
} finally {
cleanupTempFiles(workingDirectory, inputPath, outputPath);
}
@@ -2,6 +2,7 @@ package stirling.software.SPDF.controller.api.converters;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.util.Locale;
import org.jetbrains.annotations.NotNull;
@@ -10,6 +11,7 @@ import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import org.springframework.web.util.HtmlUtils;
import io.github.pixee.security.Filenames;
@@ -26,6 +28,7 @@ import stirling.software.common.model.api.converters.EmlToPdfRequest;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.CustomHtmlSanitizer;
import stirling.software.common.util.EmlToPdf;
import stirling.software.common.util.TempFile;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
@@ -48,7 +51,8 @@ public class ConvertEmlToPDF {
+ " with extensive customization options. Features include font settings,"
+ " image constraints, display modes, attachment handling, and HTML debug"
+ " output. Input: EML or MSG file, Output: PDF or HTML file. Type: SISO")
public ResponseEntity<byte[]> convertEmlToPdf(@ModelAttribute EmlToPdfRequest request) {
public ResponseEntity<StreamingResponseBody> convertEmlToPdf(
@ModelAttribute EmlToPdfRequest request) {
MultipartFile inputFile = request.getFileInput();
String originalFilename = inputFile.getOriginalFilename();
@@ -56,22 +60,19 @@ public class ConvertEmlToPDF {
// Validate input
if (inputFile.isEmpty()) {
log.error("No file provided for EML/MSG to PDF conversion.");
return ResponseEntity.badRequest()
.body("No file provided".getBytes(StandardCharsets.UTF_8));
return errorResponse(HttpStatus.BAD_REQUEST, "No file provided");
}
if (originalFilename == null || originalFilename.trim().isEmpty()) {
log.error("Filename is null or empty.");
return ResponseEntity.badRequest()
.body("Please provide a valid filename".getBytes(StandardCharsets.UTF_8));
return errorResponse(HttpStatus.BAD_REQUEST, "Please provide a valid filename");
}
// Validate file type - support EML and MSG (Outlook) files
String lowerFilename = originalFilename.toLowerCase(Locale.ROOT);
if (!lowerFilename.endsWith(".eml") && !lowerFilename.endsWith(".msg")) {
log.error("Invalid file type for EML/MSG to PDF: {}", originalFilename);
return ResponseEntity.badRequest()
.body("Please upload a valid EML or MSG file".getBytes(StandardCharsets.UTF_8));
return errorResponse(HttpStatus.BAD_REQUEST, "Please upload a valid EML or MSG file");
}
String baseFilename = Filenames.toSimpleFileName(originalFilename); // Use Filenames utility
@@ -84,16 +85,20 @@ public class ConvertEmlToPDF {
String htmlContent =
EmlToPdf.convertEmlToHtml(fileBytes, request, customHtmlSanitizer);
log.info("Successfully converted email to HTML: {}", originalFilename);
return WebResponseUtils.bytesToWebResponse(
htmlContent.getBytes(StandardCharsets.UTF_8),
baseFilename + ".html",
MediaType.TEXT_HTML);
TempFile tempOut = tempFileManager.createManagedTempFile(".html");
try {
Files.writeString(tempOut.getPath(), htmlContent, StandardCharsets.UTF_8);
} catch (Exception ex) {
tempOut.close();
throw ex;
}
return WebResponseUtils.fileToWebResponse(
tempOut, baseFilename + ".html", MediaType.TEXT_HTML);
} catch (IOException | IllegalArgumentException e) {
log.error("HTML conversion failed for {}", originalFilename, e);
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
.body(
("HTML conversion failed: " + e.getMessage())
.getBytes(StandardCharsets.UTF_8));
return errorResponse(
HttpStatus.INTERNAL_SERVER_ERROR,
"HTML conversion failed: " + e.getMessage());
}
}
@@ -111,20 +116,25 @@ public class ConvertEmlToPDF {
if (pdfBytes == null || pdfBytes.length == 0) {
log.error("PDF conversion failed - empty output for {}", originalFilename);
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
.body(
"PDF conversion failed - empty output"
.getBytes(StandardCharsets.UTF_8));
return errorResponse(
HttpStatus.INTERNAL_SERVER_ERROR,
"PDF conversion failed - empty output");
}
log.info("Successfully converted email to PDF: {}", originalFilename);
return WebResponseUtils.bytesToWebResponse(
pdfBytes, baseFilename + ".pdf", MediaType.APPLICATION_PDF);
TempFile tempOut = tempFileManager.createManagedTempFile(".pdf");
try {
Files.write(tempOut.getPath(), pdfBytes);
} catch (Exception ex) {
tempOut.close();
throw ex;
}
return WebResponseUtils.pdfFileToWebResponse(tempOut, baseFilename + ".pdf");
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
log.error("Email to PDF conversion was interrupted for {}", originalFilename, e);
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
.body("Conversion was interrupted".getBytes(StandardCharsets.UTF_8));
return errorResponse(
HttpStatus.INTERNAL_SERVER_ERROR, "Conversion was interrupted");
} catch (IllegalArgumentException e) {
String errorMessage = buildErrorMessage(e, originalFilename);
log.error(
@@ -132,8 +142,7 @@ public class ConvertEmlToPDF {
originalFilename,
errorMessage,
e);
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
.body(errorMessage.getBytes(StandardCharsets.UTF_8));
return errorResponse(HttpStatus.INTERNAL_SERVER_ERROR, errorMessage);
} catch (RuntimeException e) {
String errorMessage = buildErrorMessage(e, originalFilename);
log.error(
@@ -141,17 +150,25 @@ public class ConvertEmlToPDF {
originalFilename,
errorMessage,
e);
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
.body(errorMessage.getBytes(StandardCharsets.UTF_8));
return errorResponse(HttpStatus.INTERNAL_SERVER_ERROR, errorMessage);
}
} catch (IOException e) {
log.error("File processing error for email to PDF: {}", originalFilename, e);
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
.body("File processing error".getBytes(StandardCharsets.UTF_8));
return errorResponse(HttpStatus.INTERNAL_SERVER_ERROR, "File processing error");
}
}
private ResponseEntity<StreamingResponseBody> errorResponse(HttpStatus status, String message) {
byte[] body = message.getBytes(StandardCharsets.UTF_8);
StreamingResponseBody streaming =
os -> {
os.write(body);
os.flush();
};
return ResponseEntity.status(status).body(streaming);
}
private static @NotNull String buildErrorMessage(Exception e, String originalFilename) {
String safeFilename = HtmlUtils.htmlEscape(originalFilename);
String exceptionMessage = e.getMessage();
@@ -1,9 +1,12 @@
package stirling.software.SPDF.controller.api.converters;
import java.nio.file.Files;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import io.github.pixee.security.Filenames;
import io.swagger.v3.oas.annotations.Operation;
@@ -37,7 +40,7 @@ public class ConvertHtmlToPDF {
description =
"This endpoint takes an HTML or ZIP file input and converts it to a PDF format."
+ " Input:HTML Output:PDF Type:SISO")
public ResponseEntity<byte[]> HtmlToPdf(@ModelAttribute HTMLToPdfRequest request)
public ResponseEntity<StreamingResponseBody> HtmlToPdf(@ModelAttribute HTMLToPdfRequest request)
throws Exception {
MultipartFile fileInput = request.getFileInput();
@@ -65,6 +68,13 @@ public class ConvertHtmlToPDF {
String outputFilename = GeneralUtils.generateFilename(originalFilename, ".pdf");
return WebResponseUtils.bytesToWebResponse(pdfBytes, outputFilename);
TempFile tempOut = tempFileManager.createManagedTempFile(".pdf");
try {
Files.write(tempOut.getPath(), pdfBytes);
} catch (Exception e) {
tempOut.close();
throw e;
}
return WebResponseUtils.pdfFileToWebResponse(tempOut, outputFilename);
}
}
@@ -1,5 +1,6 @@
package stirling.software.SPDF.controller.api.converters;
import java.nio.charset.StandardCharsets;
import java.util.List;
import java.util.Map;
@@ -14,6 +15,7 @@ import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import io.github.pixee.security.Filenames;
import io.swagger.v3.oas.annotations.Operation;
@@ -46,8 +48,8 @@ public class ConvertMarkdownToPdf {
description =
"This endpoint takes a Markdown file or ZIP (containing Markdown + images) input, converts it to HTML, and then to"
+ " PDF format. Input:MARKDOWN Output:PDF Type:SISO")
public ResponseEntity<byte[]> markdownToPdf(@ModelAttribute GeneralFile generalFile)
throws Exception {
public ResponseEntity<StreamingResponseBody> markdownToPdf(
@ModelAttribute GeneralFile generalFile) throws Exception {
MultipartFile fileInput = generalFile.getFileInput();
if (fileInput == null) {
@@ -79,7 +81,7 @@ public class ConvertMarkdownToPdf {
java.nio.file.Path tempDirPath = tempDir.getPath();
try (java.util.zip.ZipInputStream zipIn =
io.github.pixee.security.ZipSecurity.createHardenedInputStream(
new java.io.ByteArrayInputStream(fileInput.getBytes()))) {
fileInput.getInputStream())) {
java.util.zip.ZipEntry entry;
while ((entry = zipIn.getNextEntry()) != null) {
if (!entry.isDirectory()) {
@@ -141,7 +143,7 @@ public class ConvertMarkdownToPdf {
List<Extension> extensions = List.of(TablesExtension.create());
Parser parser = Parser.builder().extensions(extensions).build();
Node document = parser.parse(new String(fileInput.getBytes()));
Node document = parser.parse(new String(fileInput.getBytes(), StandardCharsets.UTF_8));
HtmlRenderer renderer =
HtmlRenderer.builder()
.attributeProviderFactory(context -> new TableAttributeProvider())
@@ -154,7 +156,7 @@ public class ConvertMarkdownToPdf {
FileToPdf.convertHtmlToPdf(
runtimePathConfig.getWeasyPrintPath(),
null,
htmlContent.getBytes(),
htmlContent.getBytes(StandardCharsets.UTF_8),
"converted.html",
tempFileManager,
customHtmlSanitizer);
@@ -163,7 +165,15 @@ public class ConvertMarkdownToPdf {
}
pdfBytes = pdfDocumentFactory.createNewBytesBasedOnOldDocument(pdfBytes);
return WebResponseUtils.bytesToWebResponse(pdfBytes, outputFilename);
TempFile tempOut = tempFileManager.createManagedTempFile(".pdf");
try {
java.nio.file.Files.write(tempOut.getPath(), pdfBytes);
} catch (Exception e) {
tempOut.close();
throw e;
}
return WebResponseUtils.pdfFileToWebResponse(tempOut, outputFilename);
}
/**
@@ -17,6 +17,7 @@ import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import io.github.pixee.security.Filenames;
import io.swagger.v3.oas.annotations.Operation;
@@ -36,6 +37,8 @@ import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.ProcessExecutor;
import stirling.software.common.util.ProcessExecutor.ProcessExecutorResult;
import stirling.software.common.util.RegexPatternUtils;
import stirling.software.common.util.TempFile;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
@ConvertApi
@@ -47,6 +50,7 @@ public class ConvertOfficeController {
private final RuntimePathConfig runtimePathConfig;
private final CustomHtmlSanitizer customHtmlSanitizer;
private final EndpointConfiguration endpointConfiguration;
private final TempFileManager tempFileManager;
private boolean isUnoconvertAvailable() {
return endpointConfiguration.isGroupEnabled("Unoconvert")
@@ -202,21 +206,32 @@ public class ConvertOfficeController {
description =
"This endpoint converts a given file to a PDF using LibreOffice API Input:ANY"
+ " Output:PDF Type:SISO")
public ResponseEntity<byte[]> processFileToPDF(@ModelAttribute GeneralFile generalFile)
throws Exception {
public ResponseEntity<StreamingResponseBody> processFileToPDF(
@ModelAttribute GeneralFile generalFile) throws Exception {
MultipartFile inputFile = generalFile.getFileInput();
// unused but can start server instance if startup time is to long
// LibreOfficeListener.getInstance().start();
File file = null;
TempFile tempOut = null;
try {
file = convertToPdf(inputFile);
tempOut = tempFileManager.createManagedTempFile(".pdf");
try (PDDocument doc = pdfDocumentFactory.load(file)) {
return WebResponseUtils.pdfDocToWebResponse(
doc,
GeneralUtils.generateFilename(
inputFile.getOriginalFilename(), "_convertedToPDF.pdf"));
doc.save(tempOut.getFile());
}
String filename =
GeneralUtils.generateFilename(
inputFile.getOriginalFilename(), "_convertedToPDF.pdf");
ResponseEntity<StreamingResponseBody> response =
WebResponseUtils.pdfFileToWebResponse(tempOut, filename);
tempOut = null;
return response;
} catch (Exception e) {
if (tempOut != null) {
tempOut.close();
}
throw e;
} finally {
if (file != null && file.getParent() != null) {
FileUtils.deleteDirectory(file.getParentFile());
@@ -13,6 +13,7 @@ import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import io.github.pixee.security.Filenames;
import io.swagger.v3.oas.annotations.Operation;
@@ -29,6 +30,7 @@ import stirling.software.common.annotations.api.ConvertApi;
import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.ProcessExecutor;
import stirling.software.common.util.ProcessExecutor.ProcessExecutorResult;
import stirling.software.common.util.TempFile;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
@@ -85,8 +87,8 @@ public class ConvertPDFToEpubController {
description =
"Convert a PDF file to a high-quality EPUB or AZW3 ebook using Calibre. Input:PDF"
+ " Output:EPUB/AZW3 Type:SISO")
public ResponseEntity<byte[]> convertPdfToEpub(@ModelAttribute ConvertPdfToEpubRequest request)
throws Exception {
public ResponseEntity<StreamingResponseBody> convertPdfToEpub(
@ModelAttribute ConvertPdfToEpubRequest request) throws Exception {
if (!endpointConfiguration.isGroupEnabled(CALIBRE_GROUP)) {
throw new IllegalStateException(
@@ -170,9 +172,16 @@ public class ConvertPDFToEpubController {
+ "."
+ outputFormat.getExtension());
byte[] outputBytes = Files.readAllBytes(outputPath);
MediaType mediaType = MediaType.valueOf(outputFormat.getMediaType());
return WebResponseUtils.bytesToWebResponse(outputBytes, outputFilename, mediaType);
TempFile tempOut =
tempFileManager.createManagedTempFile("." + outputFormat.getExtension());
try {
Files.copy(outputPath, tempOut.getPath(), StandardCopyOption.REPLACE_EXISTING);
} catch (Exception e) {
tempOut.close();
throw e;
}
return WebResponseUtils.fileToWebResponse(tempOut, outputFilename, mediaType);
} finally {
cleanupTempFiles(workingDirectory, inputPath, outputPath);
}
@@ -1,6 +1,7 @@
package stirling.software.SPDF.controller.api.converters;
import java.io.ByteArrayOutputStream;
import java.io.OutputStream;
import java.nio.file.Files;
import java.util.List;
import java.util.Locale;
@@ -11,11 +12,10 @@ import org.apache.poi.ss.usermodel.Sheet;
import org.apache.poi.ss.usermodel.Workbook;
import org.apache.poi.ss.util.WorkbookUtil;
import org.apache.poi.xssf.usermodel.XSSFWorkbook;
import org.springframework.http.ContentDisposition;
import org.springframework.http.HttpHeaders;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import io.swagger.v3.oas.annotations.Operation;
@@ -27,6 +27,9 @@ import stirling.software.common.annotations.AutoJobPostMapping;
import stirling.software.common.annotations.api.ConvertApi;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.TempFile;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
import technology.tabula.ObjectExtractor;
import technology.tabula.Page;
@@ -40,6 +43,7 @@ import technology.tabula.extractors.SpreadsheetExtractionAlgorithm;
public class ConvertPDFToExcelController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final TempFileManager tempFileManager;
@AutoJobPostMapping(value = "/pdf/xlsx", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
@Operation(
@@ -47,11 +51,12 @@ public class ConvertPDFToExcelController {
description =
"Extracts tabular data from each page of a PDF and writes it into an Excel"
+ " workbook, one sheet per table. Input:PDF Output:XLSX Type:SISO")
public ResponseEntity<byte[]> pdfToExcel(@ModelAttribute PDFWithPageNums request)
public ResponseEntity<StreamingResponseBody> pdfToExcel(@ModelAttribute PDFWithPageNums request)
throws Exception {
String baseName =
GeneralUtils.removeExtension(request.getFileInput().getOriginalFilename());
TempFile tempOut = tempFileManager.createManagedTempFile(".xlsx");
try (PDDocument document = pdfDocumentFactory.load(request);
XSSFWorkbook workbook = new XSSFWorkbook();
ObjectExtractor extractor = new ObjectExtractor(document)) {
@@ -89,21 +94,22 @@ public class ConvertPDFToExcelController {
}
if (sheetCount == 0) {
tempOut.close();
return ResponseEntity.noContent().build();
}
ByteArrayOutputStream baos = new ByteArrayOutputStream();
workbook.write(baos);
HttpHeaders headers = new HttpHeaders();
headers.setContentDisposition(
ContentDisposition.builder("attachment").filename(baseName + ".xlsx").build());
headers.setContentType(
MediaType.parseMediaType(
"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"));
return ResponseEntity.ok().headers(headers).body(baos.toByteArray());
try (OutputStream os = Files.newOutputStream(tempOut.getPath())) {
workbook.write(os);
}
} catch (Exception e) {
tempOut.close();
throw e;
}
MediaType mediaType =
MediaType.parseMediaType(
"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet");
return WebResponseUtils.fileToWebResponse(tempOut, baseName + ".xlsx", mediaType);
}
private String getUniqueSheetName(Workbook workbook, String baseName) {
@@ -4,6 +4,7 @@ import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import io.swagger.v3.oas.annotations.Operation;
@@ -28,7 +29,8 @@ public class ConvertPDFToHtml {
summary = "Convert PDF to HTML",
description =
"This endpoint converts a PDF file to HTML format. Input:PDF Output:HTML Type:SISO")
public ResponseEntity<byte[]> processPdfToHTML(@ModelAttribute PDFFile file) throws Exception {
public ResponseEntity<StreamingResponseBody> processPdfToHTML(@ModelAttribute PDFFile file)
throws Exception {
MultipartFile inputFile = file.getFileInput();
PDFToFile pdfToFile = new PDFToFile(tempFileManager, runtimePathConfig);
return pdfToFile.processPdfToHtml(inputFile);
@@ -1,6 +1,8 @@
package stirling.software.SPDF.controller.api.converters;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.text.PDFTextStripper;
@@ -8,6 +10,7 @@ import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import io.swagger.v3.oas.annotations.Operation;
@@ -23,6 +26,7 @@ import stirling.software.common.model.api.PDFFile;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.PDFToFile;
import stirling.software.common.util.TempFile;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
@@ -40,7 +44,7 @@ public class ConvertPDFToOffice {
description =
"This endpoint converts a given PDF file to a Presentation format. Input:PDF"
+ " Output:PPT Type:SISO")
public ResponseEntity<byte[]> processPdfToPresentation(
public ResponseEntity<StreamingResponseBody> processPdfToPresentation(
@ModelAttribute PdfToPresentationRequest request)
throws IOException, InterruptedException {
MultipartFile inputFile = request.getFileInput();
@@ -55,20 +59,24 @@ public class ConvertPDFToOffice {
description =
"This endpoint converts a given PDF file to Text or RTF format. Input:PDF"
+ " Output:TXT Type:SISO")
public ResponseEntity<byte[]> processPdfToRTForTXT(
public ResponseEntity<StreamingResponseBody> processPdfToRTForTXT(
@ModelAttribute PdfToTextOrRTFRequest request)
throws IOException, InterruptedException {
MultipartFile inputFile = request.getFileInput();
String outputFormat = request.getOutputFormat();
if ("txt".equals(request.getOutputFormat())) {
String fileName =
GeneralUtils.generateFilename(inputFile.getOriginalFilename(), ".txt");
TempFile finalOut = tempFileManager.createManagedTempFile(".txt");
try (PDDocument document = pdfDocumentFactory.load(inputFile)) {
PDFTextStripper stripper = new PDFTextStripper();
String text = stripper.getText(document);
return WebResponseUtils.bytesToWebResponse(
text.getBytes(),
GeneralUtils.generateFilename(inputFile.getOriginalFilename(), ".txt"),
MediaType.TEXT_PLAIN);
Files.writeString(finalOut.getPath(), text, StandardCharsets.UTF_8);
} catch (Exception e) {
finalOut.close();
throw e;
}
return WebResponseUtils.fileToWebResponse(finalOut, fileName, MediaType.TEXT_PLAIN);
} else {
PDFToFile pdfToFile = new PDFToFile(tempFileManager, runtimePathConfig);
return pdfToFile.processPdfToOfficeFormat(inputFile, outputFormat, "writer_pdf_import");
@@ -81,8 +89,8 @@ public class ConvertPDFToOffice {
description =
"This endpoint converts a given PDF file to a Word document format. Input:PDF"
+ " Output:WORD Type:SISO")
public ResponseEntity<byte[]> processPdfToWord(@ModelAttribute PdfToWordRequest request)
throws IOException, InterruptedException {
public ResponseEntity<StreamingResponseBody> processPdfToWord(
@ModelAttribute PdfToWordRequest request) throws IOException, InterruptedException {
MultipartFile inputFile = request.getFileInput();
String outputFormat = request.getOutputFormat();
PDFToFile pdfToFile = new PDFToFile(tempFileManager, runtimePathConfig);
@@ -95,7 +103,8 @@ public class ConvertPDFToOffice {
description =
"This endpoint converts a PDF file to an XML file. Input:PDF Output:XML"
+ " Type:SISO")
public ResponseEntity<byte[]> processPdfToXML(@ModelAttribute PDFFile file) throws Exception {
public ResponseEntity<StreamingResponseBody> processPdfToXML(@ModelAttribute PDFFile file)
throws Exception {
MultipartFile inputFile = file.getFileInput();
PDFToFile pdfToFile = new PDFToFile(tempFileManager, runtimePathConfig);
@@ -77,6 +77,7 @@ import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.server.ResponseStatusException;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import io.github.pixee.security.Filenames;
import io.swagger.v3.oas.annotations.Operation;
@@ -92,6 +93,8 @@ import stirling.software.common.configuration.RuntimePathConfig;
import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.ProcessExecutor;
import stirling.software.common.util.ProcessExecutor.ProcessExecutorResult;
import stirling.software.common.util.TempFile;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
@ConvertApi
@@ -102,6 +105,7 @@ public class ConvertPDFToPDFA {
private static final Pattern NON_PRINTABLE_ASCII = Pattern.compile("[^\\x20-\\x7E]");
private final RuntimePathConfig runtimePathConfig;
private final stirling.software.SPDF.service.VeraPDFService veraPDFService;
private final TempFileManager tempFileManager;
private static final String ICC_RESOURCE_PATH = "/icc/sRGB2014.icc";
private static final int PDFA_COMPATIBILITY_POLICY = 1;
@@ -573,7 +577,7 @@ public class ConvertPDFToPDFA {
summary = "Convert a PDF to a PDF/A or PDF/X",
description =
"This endpoint converts a PDF file to a PDF/A or PDF/X file using Ghostscript (preferred) or PDFBox/LibreOffice (fallback). PDF/A is a format designed for long-term archiving, while PDF/X is optimized for print production. Input:PDF Output:PDF Type:SISO")
public ResponseEntity<byte[]> pdfToPdfA(@ModelAttribute PdfToPdfARequest request)
public ResponseEntity<StreamingResponseBody> pdfToPdfA(@ModelAttribute PdfToPdfARequest request)
throws Exception {
MultipartFile inputFile = request.getFileInput();
String outputFormat = request.getOutputFormat();
@@ -609,7 +613,7 @@ public class ConvertPDFToPDFA {
return missing;
}
private ResponseEntity<byte[]> handlePdfXConversion(
private ResponseEntity<StreamingResponseBody> handlePdfXConversion(
MultipartFile inputFile, String outputFormat) throws Exception {
PdfXProfile profile = PdfXProfile.fromRequest(outputFormat);
@@ -640,8 +644,14 @@ public class ConvertPDFToPDFA {
log.info("PDF/X conversion completed successfully to {}", profile.getDisplayName());
return WebResponseUtils.bytesToWebResponse(
converted, outputFilename, MediaType.APPLICATION_PDF);
TempFile tempOut = tempFileManager.createManagedTempFile(".pdf");
try {
Files.write(tempOut.getPath(), converted);
} catch (Exception ex) {
tempOut.close();
throw ex;
}
return WebResponseUtils.pdfFileToWebResponse(tempOut, outputFilename);
} catch (IOException | InterruptedException e) {
log.error("PDF/X conversion failed", e);
@@ -1796,7 +1806,7 @@ public class ConvertPDFToPDFA {
return Files.readAllBytes(outputPdf);
}
private ResponseEntity<byte[]> handlePdfAConversion(
private ResponseEntity<StreamingResponseBody> handlePdfAConversion(
MultipartFile inputFile, String outputFormat, boolean strict) throws Exception {
PdfaProfile profile = PdfaProfile.fromRequest(outputFormat);
@@ -1830,8 +1840,14 @@ public class ConvertPDFToPDFA {
verifyStrictCompliance(converted);
}
return WebResponseUtils.bytesToWebResponse(
converted, outputFilename, MediaType.APPLICATION_PDF);
TempFile tempOut = tempFileManager.createManagedTempFile(".pdf");
try {
Files.write(tempOut.getPath(), converted);
} catch (Exception ex) {
tempOut.close();
throw ex;
}
return WebResponseUtils.pdfFileToWebResponse(tempOut, outputFilename);
} catch (IOException | InterruptedException e) {
log.warn(
"Ghostscript conversion failed, falling back to PDFBox/LibreOffice method",
@@ -1851,8 +1867,14 @@ public class ConvertPDFToPDFA {
verifyStrictCompliance(converted);
}
return WebResponseUtils.bytesToWebResponse(
converted, outputFilename, MediaType.APPLICATION_PDF);
TempFile tempOut = tempFileManager.createManagedTempFile(".pdf");
try {
Files.write(tempOut.getPath(), converted);
} catch (Exception ex) {
tempOut.close();
throw ex;
}
return WebResponseUtils.pdfFileToWebResponse(tempOut, outputFilename);
} finally {
deleteQuietly(workingDir);
}
@@ -1,6 +1,7 @@
package stirling.software.SPDF.controller.api.converters;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.util.Optional;
import java.util.UUID;
import java.util.regex.Pattern;
@@ -14,6 +15,7 @@ import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import io.github.pixee.security.Filenames;
import io.swagger.v3.oas.annotations.Operation;
@@ -31,6 +33,8 @@ import stirling.software.common.model.api.GeneralFile;
import stirling.software.common.model.api.PDFFile;
import stirling.software.common.service.JobOwnershipService;
import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.TempFile;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
@Slf4j
@@ -42,6 +46,7 @@ public class ConvertPdfJsonController {
private static final Pattern WHITESPACE_PATTERN = Pattern.compile("[\\r\\n\\t]+");
private static final Pattern NON_PRINTABLE_PATTERN = Pattern.compile("[^\\x20-\\x7E]");
private final PdfJsonConversionService pdfJsonConversionService;
private final TempFileManager tempFileManager;
@Autowired(required = false)
private JobOwnershipService jobOwnershipService;
@@ -51,7 +56,7 @@ public class ConvertPdfJsonController {
summary = "Convert PDF to Text Editor Format",
description =
"Extracts PDF text, fonts, and metadata into an editable JSON structure for the text editor tool. Input:PDF Output:JSON Type:SISO")
public ResponseEntity<byte[]> convertPdfToJson(
public ResponseEntity<StreamingResponseBody> convertPdfToJson(
@ModelAttribute PDFFile request,
@RequestParam(value = "lightweight", defaultValue = "false") boolean lightweight)
throws Exception {
@@ -60,6 +65,8 @@ public class ConvertPdfJsonController {
throw ExceptionUtils.createNullArgumentException("fileInput");
}
// TODO: Refactor PdfJsonConversionService to write directly to an OutputStream
// instead of returning byte[], avoiding the intermediate heap allocation + temp file write
byte[] jsonBytes = pdfJsonConversionService.convertPdfToJson(inputFile, lightweight);
logJsonResponse("pdf/text-editor", jsonBytes);
String originalName = inputFile.getOriginalFilename();
@@ -70,7 +77,14 @@ public class ConvertPdfJsonController {
.replaceFirst("")
: "document";
String docName = baseName + ".json";
return WebResponseUtils.bytesToWebResponse(jsonBytes, docName, MediaType.APPLICATION_JSON);
TempFile tempOut = tempFileManager.createManagedTempFile(".json");
try {
Files.write(tempOut.getPath(), jsonBytes);
} catch (Exception e) {
tempOut.close();
throw e;
}
return WebResponseUtils.fileToWebResponse(tempOut, docName, MediaType.APPLICATION_JSON);
}
@AutoJobPostMapping(consumes = "multipart/form-data", value = "/text-editor/pdf")
@@ -79,8 +93,8 @@ public class ConvertPdfJsonController {
summary = "Convert Text Editor Format to PDF",
description =
"Rebuilds a PDF from the editable JSON structure generated by the text editor tool. Input:JSON Output:PDF Type:SISO")
public ResponseEntity<byte[]> convertJsonToPdf(@ModelAttribute GeneralFile request)
throws Exception {
public ResponseEntity<StreamingResponseBody> convertJsonToPdf(
@ModelAttribute GeneralFile request) throws Exception {
MultipartFile jsonFile = request.getFileInput();
if (jsonFile == null) {
throw ExceptionUtils.createNullArgumentException("fileInput");
@@ -95,7 +109,14 @@ public class ConvertPdfJsonController {
.replaceFirst("")
: "document";
String docName = baseName.endsWith(".pdf") ? baseName : baseName + ".pdf";
return WebResponseUtils.bytesToWebResponse(pdfBytes, docName);
TempFile tempOut = tempFileManager.createManagedTempFile(".pdf");
try {
Files.write(tempOut.getPath(), pdfBytes);
} catch (Exception e) {
tempOut.close();
throw e;
}
return WebResponseUtils.pdfFileToWebResponse(tempOut, docName);
}
@AutoJobPostMapping(consumes = "multipart/form-data", value = "/pdf/text-editor/metadata")
@@ -105,17 +126,15 @@ public class ConvertPdfJsonController {
"Extracts document metadata, fonts, and page dimensions for the text editor tool. Caches the document for"
+ " subsequent page requests. Returns a server-generated jobId scoped to the"
+ " authenticated user. Input:PDF Output:JSON Type:SISO")
public ResponseEntity<byte[]> extractPdfMetadata(@ModelAttribute PDFFile request)
public ResponseEntity<StreamingResponseBody> extractPdfMetadata(@ModelAttribute PDFFile request)
throws Exception {
MultipartFile inputFile = request.getFileInput();
if (inputFile == null) {
throw ExceptionUtils.createNullArgumentException("fileInput");
}
// Generate server-side UUID for job
String baseJobId = UUID.randomUUID().toString();
// Scope job to authenticated user if security is enabled
String scopedJobKey = getScopedJobKey(baseJobId);
log.debug("Extracting metadata for PDF, assigned jobId: {}", scopedJobKey);
@@ -123,20 +142,27 @@ public class ConvertPdfJsonController {
byte[] jsonBytes =
pdfJsonConversionService.extractDocumentMetadata(inputFile, scopedJobKey);
logJsonResponse("pdf/text-editor/metadata", jsonBytes);
String originalName = inputFile.getOriginalFilename();
String baseName =
(originalName != null && !originalName.isBlank())
? FILE_EXTENSION_PATTERN
.matcher(Filenames.toSimpleFileName(originalName))
.replaceFirst("")
: "document";
String docName = baseName + "_metadata.json";
// Return jobId in response header for client
TempFile tempOut = tempFileManager.createManagedTempFile(".json");
try {
Files.write(tempOut.getPath(), jsonBytes);
} catch (Exception e) {
tempOut.close();
throw e;
}
return ResponseEntity.ok()
.header("X-Job-Id", scopedJobKey)
.contentType(MediaType.APPLICATION_JSON)
.body(jsonBytes);
.contentLength(java.nio.file.Files.size(tempOut.getPath()))
.body(
os -> {
try (os) {
Files.copy(tempOut.getPath(), os);
os.flush();
} finally {
tempOut.close();
}
});
}
@AutoJobPostMapping(
@@ -149,7 +175,7 @@ public class ConvertPdfJsonController {
"Applies edits for the specified pages of a cached PDF and returns an updated PDF."
+ " Requires the PDF to have been previously cached via the text editor metadata endpoint."
+ " The jobId must be obtained from the metadata extraction endpoint.")
public ResponseEntity<byte[]> exportPartialPdf(
public ResponseEntity<StreamingResponseBody> exportPartialPdf(
@PathVariable String jobId,
@RequestBody PdfJsonDocument document,
@RequestParam(value = "filename", required = false) String filename)
@@ -158,7 +184,6 @@ public class ConvertPdfJsonController {
throw ExceptionUtils.createNullArgumentException("document");
}
// Validate job ownership
validateJobAccess(jobId);
byte[] pdfBytes = pdfJsonConversionService.exportUpdatedPages(jobId, document);
@@ -173,7 +198,14 @@ public class ConvertPdfJsonController {
.filter(title -> title != null && !title.isBlank())
.orElse("document");
String docName = baseName.endsWith(".pdf") ? baseName : baseName + ".pdf";
return WebResponseUtils.bytesToWebResponse(pdfBytes, docName);
TempFile tempOut = tempFileManager.createManagedTempFile(".pdf");
try {
Files.write(tempOut.getPath(), pdfBytes);
} catch (Exception e) {
tempOut.close();
throw e;
}
return WebResponseUtils.pdfFileToWebResponse(tempOut, docName);
}
@GetMapping(value = "/pdf/text-editor/page/{jobId}/{pageNumber}")
@@ -183,16 +215,22 @@ public class ConvertPdfJsonController {
"Retrieves a single page's content from a previously cached PDF document for the text editor tool."
+ " Requires prior call to /pdf/text-editor/metadata. The jobId must belong to the"
+ " authenticated user. Output:JSON")
public ResponseEntity<byte[]> extractSinglePage(
public ResponseEntity<StreamingResponseBody> extractSinglePage(
@PathVariable String jobId, @PathVariable int pageNumber) throws Exception {
// Validate job ownership
validateJobAccess(jobId);
byte[] jsonBytes = pdfJsonConversionService.extractSinglePage(jobId, pageNumber);
logJsonResponse("pdf/text-editor/page", jsonBytes);
String docName = "page_" + pageNumber + ".json";
return WebResponseUtils.bytesToWebResponse(jsonBytes, docName, MediaType.APPLICATION_JSON);
TempFile tempOut = tempFileManager.createManagedTempFile(".json");
try {
Files.write(tempOut.getPath(), jsonBytes);
} catch (Exception e) {
tempOut.close();
throw e;
}
return WebResponseUtils.fileToWebResponse(tempOut, docName, MediaType.APPLICATION_JSON);
}
@GetMapping(value = "/pdf/text-editor/fonts/{jobId}/{pageNumber}")
@@ -202,16 +240,22 @@ public class ConvertPdfJsonController {
"Retrieves the font payloads used by a single page from a previously cached PDF document."
+ " Requires prior call to /pdf/text-editor/metadata. The jobId must belong to the"
+ " authenticated user. Output:JSON")
public ResponseEntity<byte[]> extractPageFonts(
public ResponseEntity<StreamingResponseBody> extractPageFonts(
@PathVariable String jobId, @PathVariable int pageNumber) throws Exception {
// Validate job ownership
validateJobAccess(jobId);
byte[] jsonBytes = pdfJsonConversionService.extractPageFonts(jobId, pageNumber);
logJsonResponse("pdf/text-editor/fonts/page", jsonBytes);
String docName = "page_fonts_" + pageNumber + ".json";
return WebResponseUtils.bytesToWebResponse(jsonBytes, docName, MediaType.APPLICATION_JSON);
TempFile tempOut = tempFileManager.createManagedTempFile(".json");
try {
Files.write(tempOut.getPath(), jsonBytes);
} catch (Exception e) {
tempOut.close();
throw e;
}
return WebResponseUtils.fileToWebResponse(tempOut, docName, MediaType.APPLICATION_JSON);
}
@AutoJobPostMapping(
@@ -225,24 +269,16 @@ public class ConvertPdfJsonController {
+ " authenticated user.")
public ResponseEntity<Void> clearCache(@PathVariable String jobId) {
// Validate job ownership
validateJobAccess(jobId);
pdfJsonConversionService.clearCachedDocument(jobId);
return ResponseEntity.ok().build();
}
/**
* Get a scoped job key that includes user ownership when security is enabled.
*
* @param baseJobId the base job identifier
* @return scoped job key, or just baseJobId if no ownership service available
*/
private String getScopedJobKey(String baseJobId) {
if (jobOwnershipService != null) {
return jobOwnershipService.createScopedJobKey(baseJobId);
}
// Security disabled, return unsecured job key
return baseJobId;
}
@@ -252,7 +288,6 @@ public class ConvertPdfJsonController {
return;
}
// Only perform expensive tail extraction if debug logging is enabled
if (log.isDebugEnabled()) {
int length = jsonBytes.length;
boolean endsWithJson =
@@ -431,16 +466,9 @@ public class ConvertPdfJsonController {
return WHITESPACE_PATTERN.matcher(value.substring(0, max)).replaceAll(" ") + "...";
}
/**
* Validate that the current user has access to the given job.
*
* @param jobId the job identifier to validate
* @throws SecurityException if current user does not own the job
*/
private void validateJobAccess(String jobId) {
if (jobOwnershipService != null) {
jobOwnershipService.validateJobAccess(jobId);
}
// If jobOwnershipService is null (security disabled), allow all access
}
}
@@ -8,10 +8,7 @@ import java.awt.Graphics2D;
import java.awt.geom.AffineTransform;
import java.awt.image.BufferedImage;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.text.DecimalFormat;
import java.text.DecimalFormatSymbols;
import java.util.ArrayList;
import java.util.List;
import java.util.Locale;
@@ -23,31 +20,17 @@ import javax.imageio.ImageIO;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.rendering.ImageType;
import org.apache.pdfbox.rendering.PDFRenderer;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.multipart.MultipartFile;
import io.github.pixee.security.Filenames;
import io.swagger.v3.oas.annotations.Operation;
import lombok.RequiredArgsConstructor;
import stirling.software.SPDF.model.api.converters.PdfToVideoRequest;
import stirling.software.common.annotations.AutoJobPostMapping;
import stirling.software.common.annotations.api.ConvertApi;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.ApplicationContextProvider;
import stirling.software.common.util.CheckProgramInstall;
import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.ProcessExecutor;
import stirling.software.common.util.ProcessExecutor.ProcessExecutorResult;
import stirling.software.common.util.TempDirectory;
import stirling.software.common.util.TempFile;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
@ConvertApi
@RequiredArgsConstructor
@@ -64,6 +47,8 @@ public class ConvertPdfToVideoController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final TempFileManager tempFileManager;
// ffmpeg disabled due to raised CVEs
/*
@AutoJobPostMapping(consumes = MediaType.MULTIPART_FORM_DATA_VALUE, value = "/pdf/video")
@Operation(
summary = "Convert PDF to Video Slideshow",
@@ -163,6 +148,7 @@ public class ConvertPdfToVideoController {
return WebResponseUtils.bytesToWebResponse(videoBytes, outputName, mediaType);
}
}
*/
private void generateFrames(
Path inputPdf,
@@ -14,6 +14,7 @@ import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import io.github.pixee.security.Filenames;
import io.swagger.v3.oas.annotations.Operation;
@@ -53,7 +54,8 @@ public class ConvertSvgToPDF {
+ "SVG dimensions (width/height) determine the PDF page size; defaults to A4 if not specified. "
+ "SVG content is sanitized to prevent XSS attacks. "
+ "Input: SVG file(s), Output: PDF file(s) or ZIP. Type: MIMO")
public ResponseEntity<byte[]> convertSvgToPdf(@ModelAttribute SvgToPdfRequest request) {
public ResponseEntity<StreamingResponseBody> convertSvgToPdf(
@ModelAttribute SvgToPdfRequest request) {
MultipartFile[] inputFiles = request.getFileInput();
boolean combineIntoSinglePdf = Boolean.TRUE.equals(request.getCombineIntoSinglePdf());
@@ -61,8 +63,7 @@ public class ConvertSvgToPDF {
// Validate input
if (inputFiles == null || inputFiles.length == 0) {
log.error("No files provided for SVG to PDF conversion.");
return ResponseEntity.badRequest()
.body("No files provided".getBytes(StandardCharsets.UTF_8));
return errorResponse(HttpStatus.BAD_REQUEST, "No files provided");
}
try {
@@ -103,8 +104,7 @@ public class ConvertSvgToPDF {
if (sanitizedSvgs.isEmpty()) {
log.error("No valid SVG files were found");
return ResponseEntity.status(HttpStatus.BAD_REQUEST)
.body("No valid SVG files were found".getBytes(StandardCharsets.UTF_8));
return errorResponse(HttpStatus.BAD_REQUEST, "No valid SVG files were found");
}
if (combineIntoSinglePdf) {
@@ -115,14 +115,23 @@ public class ConvertSvgToPDF {
} catch (Exception e) {
log.error("Unexpected error during SVG to PDF conversion", e);
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
.body(
"An unexpected error occurred during conversion"
.getBytes(StandardCharsets.UTF_8));
return errorResponse(
HttpStatus.INTERNAL_SERVER_ERROR,
"An unexpected error occurred during conversion");
}
}
private ResponseEntity<byte[]> handleCombinedConversion(
private ResponseEntity<StreamingResponseBody> errorResponse(HttpStatus status, String message) {
byte[] body = message.getBytes(StandardCharsets.UTF_8);
StreamingResponseBody streaming =
os -> {
os.write(body);
os.flush();
};
return ResponseEntity.status(status).body(streaming);
}
private ResponseEntity<StreamingResponseBody> handleCombinedConversion(
List<byte[]> sanitizedSvgs, List<String> filenames) {
try {
log.info("Combining {} SVG files into single PDF", sanitizedSvgs.size());
@@ -131,10 +140,8 @@ public class ConvertSvgToPDF {
if (pdfBytes == null || pdfBytes.length == 0) {
log.error("PDF conversion failed - empty output");
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
.body(
"PDF conversion failed - empty output"
.getBytes(StandardCharsets.UTF_8));
return errorResponse(
HttpStatus.INTERNAL_SERVER_ERROR, "PDF conversion failed - empty output");
}
pdfBytes = pdfDocumentFactory.createNewBytesBasedOnOldDocument(pdfBytes);
@@ -146,19 +153,23 @@ public class ConvertSvgToPDF {
log.info("Successfully combined {} SVGs into single PDF", sanitizedSvgs.size());
return WebResponseUtils.bytesToWebResponse(
pdfBytes, outputFilename, MediaType.APPLICATION_PDF);
TempFile tempOut = tempFileManager.createManagedTempFile(".pdf");
try {
Files.write(tempOut.getPath(), pdfBytes);
} catch (Exception e) {
tempOut.close();
throw e;
}
return WebResponseUtils.pdfFileToWebResponse(tempOut, outputFilename);
} catch (IOException e) {
log.error("Error combining SVGs into PDF", e);
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
.body(
("Conversion failed: " + e.getMessage())
.getBytes(StandardCharsets.UTF_8));
return errorResponse(
HttpStatus.INTERNAL_SERVER_ERROR, "Conversion failed: " + e.getMessage());
}
}
private ResponseEntity<byte[]> handleSeparateConversion(
private ResponseEntity<StreamingResponseBody> handleSeparateConversion(
List<byte[]> sanitizedSvgs, List<String> filenames) {
List<ConvertedPdf> convertedPdfs = new ArrayList<>();
@@ -188,15 +199,21 @@ public class ConvertSvgToPDF {
if (convertedPdfs.isEmpty()) {
log.error("No files were successfully converted");
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
.body("No files were successfully converted".getBytes(StandardCharsets.UTF_8));
return errorResponse(
HttpStatus.INTERNAL_SERVER_ERROR, "No files were successfully converted");
}
try {
if (convertedPdfs.size() == 1) {
ConvertedPdf pdf = convertedPdfs.get(0);
return WebResponseUtils.bytesToWebResponse(
pdf.content, pdf.filename, MediaType.APPLICATION_PDF);
TempFile tempOut = tempFileManager.createManagedTempFile(".pdf");
try {
Files.write(tempOut.getPath(), pdf.content);
} catch (Exception e) {
tempOut.close();
throw e;
}
return WebResponseUtils.pdfFileToWebResponse(tempOut, pdf.filename);
}
String zipFilename =
@@ -204,22 +221,18 @@ public class ConvertSvgToPDF {
? "converted_svgs.zip"
: GeneralUtils.generateFilename(
filenames.get(0), "_converted_svgs.zip");
byte[] zipBytes = createZipFromPdfs(convertedPdfs);
return WebResponseUtils.bytesToWebResponse(
zipBytes, zipFilename, MediaType.APPLICATION_OCTET_STREAM);
TempFile zipFile = createZipFromPdfs(convertedPdfs);
return WebResponseUtils.zipFileToWebResponse(zipFile, zipFilename);
} catch (IOException e) {
log.error("Failed to create response", e);
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
.body("Failed to create response".getBytes(StandardCharsets.UTF_8));
return errorResponse(HttpStatus.INTERNAL_SERVER_ERROR, "Failed to create response");
}
}
private byte[] createZipFromPdfs(List<ConvertedPdf> pdfs) throws IOException {
try (TempFile tempZipFile = new TempFile(tempFileManager, ".zip");
ZipOutputStream zipOut =
new ZipOutputStream(Files.newOutputStream(tempZipFile.getPath()))) {
private TempFile createZipFromPdfs(List<ConvertedPdf> pdfs) throws IOException {
TempFile tempZipFile = tempFileManager.createManagedTempFile(".zip");
try (ZipOutputStream zipOut =
new ZipOutputStream(Files.newOutputStream(tempZipFile.getPath()))) {
for (ConvertedPdf pdf : pdfs) {
ZipEntry pdfEntry = new ZipEntry(pdf.filename);
zipOut.putNextEntry(pdfEntry);
@@ -227,9 +240,11 @@ public class ConvertSvgToPDF {
zipOut.closeEntry();
log.debug("Added {} to ZIP", pdf.filename);
}
return Files.readAllBytes(tempZipFile.getPath());
} catch (IOException e) {
tempZipFile.close();
throw e;
}
return tempZipFile;
}
private static class ConvertedPdf {
@@ -1,6 +1,5 @@
package stirling.software.SPDF.controller.api.converters;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.net.URI;
import java.net.http.HttpClient;
@@ -39,6 +38,8 @@ import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.ProcessExecutor;
import stirling.software.common.util.RegexPatternUtils;
import stirling.software.common.util.TempFile;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
@ConvertApi
@@ -49,6 +50,7 @@ public class ConvertWebsiteToPDF {
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final RuntimePathConfig runtimePathConfig;
private final ApplicationProperties applicationProperties;
private final TempFileManager tempFileManager;
private static final Pattern FILE_SCHEME_PATTERN =
Pattern.compile("(?<![a-z0-9_])file\\s*:(?:/{1,3}|%2f|%5c|%3a|&#x2f;|&#47;)");
@@ -136,14 +138,15 @@ public class ConvertWebsiteToPDF {
.runCommandWithOutputHandling(command);
// Load the PDF using pdfDocumentFactory
try (PDDocument doc = pdfDocumentFactory.load(tempOutputFile.toFile());
ByteArrayOutputStream baos = new ByteArrayOutputStream()) {
// Convert URL to a safe filename
String outputFilename = convertURLToFileName(URL);
doc.save(baos);
return WebResponseUtils.baosToWebResponse(baos, outputFilename);
String outputFilename = convertURLToFileName(URL);
TempFile tempOut = tempFileManager.createManagedTempFile(".pdf");
try (PDDocument doc = pdfDocumentFactory.load(tempOutputFile.toFile())) {
doc.save(tempOut.getFile());
} catch (Exception e) {
tempOut.close();
throw e;
}
return WebResponseUtils.pdfFileToWebResponse(tempOut, outputFilename);
} finally {
if (tempHtmlInput != null) {
try {
@@ -1,7 +1,6 @@
package stirling.software.SPDF.controller.api.converters;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.StringWriter;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
@@ -32,6 +31,7 @@ import stirling.software.common.annotations.AutoJobPostMapping;
import stirling.software.common.annotations.api.ConvertApi;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.WebResponseUtils;
import technology.tabula.ObjectExtractor;
import technology.tabula.Page;
@@ -90,7 +90,7 @@ public class ExtractCSVController {
}
private ResponseEntity<byte[]> createZipResponse(List<CsvEntry> entries, String baseName)
throws IOException {
throws Exception {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
try (ZipOutputStream zipOut = new ZipOutputStream(baos)) {
for (CsvEntry entry : entries) {
@@ -101,14 +101,10 @@ public class ExtractCSVController {
}
}
HttpHeaders headers = new HttpHeaders();
headers.setContentDisposition(
ContentDisposition.builder("attachment")
.filename(baseName + "_extracted.zip")
.build());
headers.setContentType(MediaType.parseMediaType("application/zip"));
return ResponseEntity.ok().headers(headers).body(baos.toByteArray());
return WebResponseUtils.bytesToWebResponse(
baos.toByteArray(),
baseName + "_extracted.zip",
MediaType.APPLICATION_OCTET_STREAM);
}
private ResponseEntity<String> createCsvResponse(CsvEntry entry, String baseName) {
@@ -13,6 +13,7 @@ import org.apache.commons.io.FilenameUtils;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import io.swagger.v3.oas.annotations.Operation;
@@ -38,7 +39,6 @@ import stirling.software.common.util.WebResponseUtils;
@RequiredArgsConstructor
public class PdfVectorExportController {
private static final MediaType PDF_MEDIA_TYPE = MediaType.APPLICATION_PDF;
private static final Set<String> GHOSTSCRIPT_INPUTS =
Set.of("ps", "eps", "epsf"); // PCL/PXL/XPS require GhostPDL (gpcl6/gxps)
@@ -51,7 +51,7 @@ public class PdfVectorExportController {
description =
"Converts PostScript vector inputs (PS, EPS, EPSF) to PDF using Ghostscript."
+ " Input:PS/EPS Output:PDF Type:SISO")
public ResponseEntity<byte[]> convertGhostscriptInputsToPdf(
public ResponseEntity<StreamingResponseBody> convertGhostscriptInputsToPdf(
@Valid @ModelAttribute PdfVectorExportRequest request) throws Exception {
String originalName =
@@ -63,9 +63,9 @@ public class PdfVectorExportController {
? FilenameUtils.getExtension(originalName).toLowerCase(Locale.ROOT)
: "";
TempFile outputTemp = tempFileManager.createManagedTempFile(".pdf");
try (TempFile inputTemp =
new TempFile(tempFileManager, extension.isEmpty() ? "" : "." + extension);
TempFile outputTemp = new TempFile(tempFileManager, ".pdf")) {
new TempFile(tempFileManager, extension.isEmpty() ? "" : "." + extension)) {
request.getFileInput().transferTo(inputTemp.getFile());
@@ -83,11 +83,13 @@ public class PdfVectorExportController {
"Unsupported Ghostscript input format {0}",
extension);
}
byte[] pdfBytes = Files.readAllBytes(outputTemp.getPath());
String outputName = GeneralUtils.generateFilename(originalName, "_converted.pdf");
return WebResponseUtils.bytesToWebResponse(pdfBytes, outputName, PDF_MEDIA_TYPE);
} catch (Exception e) {
outputTemp.close();
throw e;
}
String outputName = GeneralUtils.generateFilename(originalName, "_converted.pdf");
return WebResponseUtils.pdfFileToWebResponse(outputTemp, outputName);
}
@AutoJobPostMapping(consumes = MediaType.MULTIPART_FORM_DATA_VALUE, value = "/pdf/vector")
@@ -96,7 +98,7 @@ public class PdfVectorExportController {
description =
"Converts PDF to Ghostscript vector formats (EPS, PS, PCL, or XPS)."
+ " Input:PDF Output:VECTOR Type:SISO")
public ResponseEntity<byte[]> convertPdfToVector(
public ResponseEntity<StreamingResponseBody> convertPdfToVector(
@Valid @ModelAttribute PdfVectorExportRequest request) throws Exception {
String originalName =
@@ -110,35 +112,37 @@ public class PdfVectorExportController {
}
outputFormat = outputFormat.toLowerCase(Locale.ROOT);
try (TempFile inputTemp = new TempFile(tempFileManager, ".pdf");
TempFile outputTemp = new TempFile(tempFileManager, "." + outputFormat)) {
TempFile outputTemp = tempFileManager.createManagedTempFile("." + outputFormat);
try (TempFile inputTemp = new TempFile(tempFileManager, ".pdf")) {
request.getFileInput().transferTo(inputTemp.getFile());
runGhostscriptPdfToVector(inputTemp.getPath(), outputTemp.getPath(), outputFormat);
byte[] vectorBytes = Files.readAllBytes(outputTemp.getPath());
String outputName =
GeneralUtils.generateFilename(originalName, "_converted." + outputFormat);
MediaType mediaType;
switch (outputFormat.toLowerCase(Locale.ROOT)) {
case "eps":
case "ps":
mediaType = MediaType.parseMediaType("application/postscript");
break;
case "pcl":
mediaType = MediaType.parseMediaType("application/vnd.hp-PCL");
break;
case "xps":
mediaType = MediaType.parseMediaType("application/vnd.ms-xpsdocument");
break;
default:
mediaType = MediaType.APPLICATION_OCTET_STREAM;
}
return WebResponseUtils.bytesToWebResponse(vectorBytes, outputName, mediaType);
} catch (Exception e) {
outputTemp.close();
throw e;
}
String outputName =
GeneralUtils.generateFilename(originalName, "_converted." + outputFormat);
MediaType mediaType;
switch (outputFormat.toLowerCase(Locale.ROOT)) {
case "eps":
case "ps":
mediaType = MediaType.parseMediaType("application/postscript");
break;
case "pcl":
mediaType = MediaType.parseMediaType("application/vnd.hp-PCL");
break;
case "xps":
mediaType = MediaType.parseMediaType("application/vnd.ms-xpsdocument");
break;
default:
mediaType = MediaType.APPLICATION_OCTET_STREAM;
}
return WebResponseUtils.fileToWebResponse(outputTemp, outputName, mediaType);
}
private void runGhostscriptPdfToVector(Path inputPath, Path outputPath, String outputFormat)
@@ -9,6 +9,7 @@ import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import io.github.pixee.security.Filenames;
import io.swagger.v3.oas.annotations.Operation;
@@ -29,6 +30,7 @@ import stirling.software.common.annotations.api.FilterApi;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.PdfUtils;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
@FilterApi
@@ -36,6 +38,7 @@ import stirling.software.common.util.WebResponseUtils;
public class FilterController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final TempFileManager tempFileManager;
@AutoJobPostMapping(
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
@@ -53,8 +56,8 @@ public class FilterController {
description = "PDF did not pass filter",
content = @Content())
})
public ResponseEntity<byte[]> containsText(@ModelAttribute ContainsTextRequest request)
throws IOException, InterruptedException {
public ResponseEntity<StreamingResponseBody> containsText(
@ModelAttribute ContainsTextRequest request) throws IOException, InterruptedException {
MultipartFile inputFile = request.getFileInput();
String text = request.getText();
String pageNumber = request.getPageNumbers();
@@ -62,7 +65,9 @@ public class FilterController {
try (PDDocument pdfDocument = pdfDocumentFactory.load(inputFile)) {
if (PdfUtils.hasText(pdfDocument, pageNumber, text)) {
return WebResponseUtils.pdfDocToWebResponse(
pdfDocument, Filenames.toSimpleFileName(inputFile.getOriginalFilename()));
pdfDocument,
Filenames.toSimpleFileName(inputFile.getOriginalFilename()),
tempFileManager);
}
}
return ResponseEntity.noContent().build();
@@ -84,15 +89,17 @@ public class FilterController {
description = "PDF did not pass filter",
content = @Content())
})
public ResponseEntity<byte[]> containsImage(@ModelAttribute PDFWithPageNums request)
throws IOException, InterruptedException {
public ResponseEntity<StreamingResponseBody> containsImage(
@ModelAttribute PDFWithPageNums request) throws IOException, InterruptedException {
MultipartFile inputFile = request.getFileInput();
String pageNumber = request.getPageNumbers();
try (PDDocument pdfDocument = pdfDocumentFactory.load(inputFile)) {
if (PdfUtils.hasImages(pdfDocument, pageNumber)) {
return WebResponseUtils.pdfDocToWebResponse(
pdfDocument, Filenames.toSimpleFileName(inputFile.getOriginalFilename()));
pdfDocument,
Filenames.toSimpleFileName(inputFile.getOriginalFilename()),
tempFileManager);
}
}
return ResponseEntity.noContent().build();
@@ -18,6 +18,7 @@ import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RequestPart;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import com.opencsv.CSVWriter;
@@ -34,6 +35,7 @@ import stirling.software.common.model.FormFieldWithCoordinates;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.FormUtils;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
import tools.jackson.core.type.TypeReference;
@@ -59,12 +61,11 @@ public class FormFillController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final ObjectMapper objectMapper;
private final TempFileManager tempFileManager;
private static ResponseEntity<byte[]> saveDocument(PDDocument document, String baseName)
private ResponseEntity<StreamingResponseBody> saveDocument(PDDocument document, String baseName)
throws IOException {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
document.save(baos);
return WebResponseUtils.bytesToWebResponse(baos.toByteArray(), baseName + ".pdf");
return WebResponseUtils.pdfDocToWebResponse(document, baseName + ".pdf", tempFileManager);
}
private static String buildBaseName(MultipartFile file, String suffix) {
@@ -261,7 +262,7 @@ public class FormFillController {
summary = "Modify existing form fields",
description =
"Updates existing fields in the provided PDF and returns the updated file")
public ResponseEntity<byte[]> modifyFields(
public ResponseEntity<StreamingResponseBody> modifyFields(
@Parameter(
description = "The input PDF file",
required = true,
@@ -292,7 +293,7 @@ public class FormFillController {
@Operation(
summary = "Delete form fields",
description = "Removes the specified fields from the PDF and returns the updated file")
public ResponseEntity<byte[]> deleteFields(
public ResponseEntity<StreamingResponseBody> deleteFields(
@Parameter(
description = "The input PDF file",
required = true,
@@ -328,7 +329,7 @@ public class FormFillController {
description =
"Populates the supplied PDF form using values from the provided JSON payload"
+ " and returns the filled PDF")
public ResponseEntity<byte[]> fillForm(
public ResponseEntity<StreamingResponseBody> fillForm(
@Parameter(
description = "The input PDF file",
required = true,
@@ -355,7 +356,7 @@ public class FormFillController {
document -> FormUtils.applyFieldValues(document, values, flatten, true));
}
private ResponseEntity<byte[]> processSingleFile(
private ResponseEntity<StreamingResponseBody> processSingleFile(
MultipartFile file, String suffix, DocumentProcessor processor) throws IOException {
requirePdf(file);
@@ -1,7 +1,7 @@
package stirling.software.SPDF.controller.api.misc;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.nio.file.Files;
import java.util.List;
import java.util.Optional;
@@ -10,6 +10,7 @@ import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import io.github.pixee.security.Filenames;
import io.swagger.v3.oas.annotations.Operation;
@@ -30,6 +31,8 @@ import stirling.software.common.annotations.api.MiscApi;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.TempFile;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
@MiscApi
@@ -43,14 +46,16 @@ public class AttachmentController {
private final ConvertPDFToPDFA convertPDFToPDFA;
private final TempFileManager tempFileManager;
@AutoJobPostMapping(consumes = MediaType.MULTIPART_FORM_DATA_VALUE, value = "/add-attachments")
@StandardPdfResponse
@Operation(
summary = "Add attachments to PDF",
description =
"This endpoint adds attachments to a PDF. Input:PDF, Output:PDF Type:MISO")
public ResponseEntity<byte[]> addAttachments(@ModelAttribute AddAttachmentRequest request)
throws Exception {
public ResponseEntity<StreamingResponseBody> addAttachments(
@ModelAttribute AddAttachmentRequest request) throws Exception {
MultipartFile fileInput = request.getFileInput();
List<MultipartFile> attachments = request.getAttachments();
boolean convertToPdfA3b = request.isConvertToPdfA3b();
@@ -79,13 +84,9 @@ public class AttachmentController {
ConvertPDFToPDFA.fixType1FontCharSet(pdfaDocument);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
pdfaDocument.save(baos);
byte[] resultBytes = baos.toByteArray();
String outputFilename = baseFileName + "_with_attachments_PDFA-3b.pdf";
return WebResponseUtils.bytesToWebResponse(
resultBytes, outputFilename, MediaType.APPLICATION_PDF);
return WebResponseUtils.pdfDocToWebResponse(
pdfaDocument, outputFilename, tempFileManager);
}
} else {
try (PDDocument document = pdfDocumentFactory.load(request, false)) {
@@ -94,7 +95,8 @@ public class AttachmentController {
document,
GeneralUtils.generateFilename(
Filenames.toSimpleFileName(fileInput.getOriginalFilename()),
"_with_attachments.pdf"));
"_with_attachments.pdf"),
tempFileManager);
}
}
}
@@ -141,7 +143,7 @@ public class AttachmentController {
description =
"This endpoint extracts all embedded attachments from a PDF into a ZIP archive."
+ " Input:PDF Output:ZIP Type:SISO")
public ResponseEntity<byte[]> extractAttachments(
public ResponseEntity<StreamingResponseBody> extractAttachments(
@ModelAttribute ExtractAttachmentsRequest request) throws IOException {
try (PDDocument document = pdfDocumentFactory.load(request, true)) {
Optional<byte[]> extracted = pdfAttachmentService.extractAttachments(document);
@@ -159,8 +161,14 @@ public class AttachmentController {
Filenames.toSimpleFileName(
GeneralUtils.generateFilename(sourceName, "_attachments.zip"));
return WebResponseUtils.bytesToWebResponse(
extracted.get(), outputName, MediaType.APPLICATION_OCTET_STREAM);
TempFile tempOut = tempFileManager.createManagedTempFile(".zip");
try {
Files.write(tempOut.getFile().toPath(), extracted.get());
} catch (IOException e) {
tempOut.close();
throw e;
}
return WebResponseUtils.zipFileToWebResponse(tempOut, outputName);
}
}
@@ -187,8 +195,8 @@ public class AttachmentController {
summary = "Rename attachment in PDF",
description =
"This endpoint renames an embedded attachment in a PDF. Input:PDF Output:PDF Type:MISO")
public ResponseEntity<byte[]> renameAttachment(@ModelAttribute RenameAttachmentRequest request)
throws Exception {
public ResponseEntity<StreamingResponseBody> renameAttachment(
@ModelAttribute RenameAttachmentRequest request) throws Exception {
MultipartFile fileInput = request.getFileInput();
String attachmentName = request.getAttachmentName();
String newName = request.getNewName();
@@ -209,7 +217,8 @@ public class AttachmentController {
document,
GeneralUtils.generateFilename(
Filenames.toSimpleFileName(fileInput.getOriginalFilename()),
"_attachment_renamed.pdf"));
"_attachment_renamed.pdf"),
tempFileManager);
}
}
@@ -221,8 +230,8 @@ public class AttachmentController {
summary = "Delete attachment from PDF",
description =
"This endpoint deletes an embedded attachment from a PDF. Input:PDF Output:PDF Type:MISO")
public ResponseEntity<byte[]> deleteAttachment(@ModelAttribute DeleteAttachmentRequest request)
throws Exception {
public ResponseEntity<StreamingResponseBody> deleteAttachment(
@ModelAttribute DeleteAttachmentRequest request) throws Exception {
MultipartFile fileInput = request.getFileInput();
String attachmentName = request.getAttachmentName();
@@ -238,7 +247,8 @@ public class AttachmentController {
document,
GeneralUtils.generateFilename(
Filenames.toSimpleFileName(fileInput.getOriginalFilename()),
"_attachment_deleted.pdf"));
"_attachment_deleted.pdf"),
tempFileManager);
}
}
}
@@ -12,6 +12,7 @@ import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import io.github.pixee.security.Filenames;
import io.swagger.v3.oas.annotations.Operation;
@@ -24,6 +25,7 @@ import stirling.software.common.annotations.AutoJobPostMapping;
import stirling.software.common.annotations.api.MiscApi;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.RegexPatternUtils;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
@MiscApi
@@ -35,6 +37,7 @@ public class AutoRenameController {
private static final int LINE_LIMIT = 200;
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final TempFileManager tempFileManager;
@AutoJobPostMapping(consumes = MediaType.MULTIPART_FORM_DATA_VALUE, value = "/auto-rename")
@Operation(
@@ -42,8 +45,8 @@ public class AutoRenameController {
description =
"This endpoint accepts a PDF file and attempts to extract its title or header"
+ " based on heuristics. Input:PDF Output:PDF Type:SISO")
public ResponseEntity<byte[]> extractHeader(@ModelAttribute ExtractHeaderRequest request)
throws Exception {
public ResponseEntity<StreamingResponseBody> extractHeader(
@ModelAttribute ExtractHeaderRequest request) throws Exception {
MultipartFile file = request.getFileInput();
boolean useFirstTextAsFallback = Boolean.TRUE.equals(request.getUseFirstTextAsFallback());
@@ -140,11 +143,14 @@ public class AutoRenameController {
.matcher(header)
.replaceAll("")
.trim();
return WebResponseUtils.pdfDocToWebResponse(document, header + ".pdf");
return WebResponseUtils.pdfDocToWebResponse(
document, header + ".pdf", tempFileManager);
} else {
log.info("File has no good title to be found");
return WebResponseUtils.pdfDocToWebResponse(
document, Filenames.toSimpleFileName(file.getOriginalFilename()));
document,
Filenames.toSimpleFileName(file.getOriginalFilename()),
tempFileManager);
}
}
}
@@ -1,26 +1,31 @@
package stirling.software.SPDF.controller.api.misc;
import java.awt.Graphics2D;
import java.awt.image.BufferedImage;
import java.awt.image.DataBufferByte;
import java.awt.image.DataBufferInt;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.nio.file.Files;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.EnumMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;
import org.apache.pdfbox.cos.COSName;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDPage;
import org.apache.pdfbox.pdmodel.graphics.image.PDImageXObject;
import org.apache.pdfbox.rendering.PDFRenderer;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import com.google.zxing.*;
import com.google.zxing.common.GlobalHistogramBinarizer;
import com.google.zxing.common.HybridBinarizer;
import io.github.pixee.security.Filenames;
@@ -35,7 +40,6 @@ import stirling.software.common.annotations.AutoJobPostMapping;
import stirling.software.common.annotations.api.MiscApi;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.ApplicationContextProvider;
import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.TempFile;
@@ -48,61 +52,219 @@ import stirling.software.common.util.WebResponseUtils;
public class AutoSplitPdfController {
private static final Set<String> VALID_QR_CONTENTS =
new HashSet<>(
Set.of(
"https://github.com/Stirling-Tools/Stirling-PDF",
"https://github.com/Frooodle/Stirling-PDF",
"https://stirlingpdf.com"));
Set.of(
"https://github.com/Stirling-Tools/Stirling-PDF",
"https://github.com/Frooodle/Stirling-PDF",
"https://stirlingpdf.com");
private static final int MAX_IMAGES_FOR_DIRECT_EXTRACTION = 3;
// 150 DPI is sufficient for QR code detection — higher wastes memory and CPU
private static final int QR_DETECTION_DPI = 150;
// Max total pixels before we downscale to avoid OOM on getRGB() allocation
private static final long MAX_IMAGE_PIXELS = 100_000_000L; // ~10000x10000
// Number of evenly-spaced pixel samples used for the blank image check
private static final int BLANK_CHECK_SAMPLES = 20;
private static final Map<DecodeHintType, Object> DECODE_HINTS;
static {
DECODE_HINTS = new EnumMap<>(DecodeHintType.class);
DECODE_HINTS.put(DecodeHintType.TRY_HARDER, Boolean.TRUE);
DECODE_HINTS.put(DecodeHintType.ALSO_INVERTED, Boolean.TRUE);
DECODE_HINTS.put(DecodeHintType.POSSIBLE_FORMATS, List.of(BarcodeFormat.QR_CODE));
}
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final TempFileManager tempFileManager;
private final ApplicationProperties applicationProperties;
private static String decodeQRCode(BufferedImage bufferedImage) {
LuminanceSource source;
if (bufferedImage.getRaster().getDataBuffer() instanceof DataBufferByte dataBufferByte) {
byte[] pixels = dataBufferByte.getData();
source =
new PlanarYUVLuminanceSource(
pixels,
bufferedImage.getWidth(),
bufferedImage.getHeight(),
0,
0,
bufferedImage.getWidth(),
bufferedImage.getHeight(),
false);
} else if (bufferedImage.getRaster().getDataBuffer()
instanceof DataBufferInt dataBufferInt) {
int[] pixels = dataBufferInt.getData();
byte[] newPixels = new byte[pixels.length];
for (int i = 0; i < pixels.length; i++) {
newPixels[i] = (byte) (pixels[i] & 0xff);
}
source =
new PlanarYUVLuminanceSource(
newPixels,
bufferedImage.getWidth(),
bufferedImage.getHeight(),
0,
0,
bufferedImage.getWidth(),
bufferedImage.getHeight(),
false);
} else {
throw new IllegalArgumentException(
"BufferedImage must have 8-bit gray scale, 24-bit RGB, 32-bit ARGB (packed"
+ " int), byte gray, or 3-byte/4-byte RGB image data");
/**
* Downscale an image if it exceeds the maximum pixel count. Scales uniformly based on the
* pixel-count ratio so both portrait and landscape images are handled correctly.
*/
private static BufferedImage downscaleIfNeeded(BufferedImage image) {
long totalPixels = (long) image.getWidth() * image.getHeight();
if (totalPixels <= MAX_IMAGE_PIXELS) {
return image;
}
double scale = Math.sqrt((double) MAX_IMAGE_PIXELS / totalPixels);
int newWidth = Math.max(1, (int) (image.getWidth() * scale));
int newHeight = Math.max(1, (int) (image.getHeight() * scale));
log.debug(
"Downscaling image from {}x{} to {}x{} for QR detection",
image.getWidth(),
image.getHeight(),
newWidth,
newHeight);
BufferedImage scaled = new BufferedImage(newWidth, newHeight, BufferedImage.TYPE_INT_RGB);
Graphics2D g = scaled.createGraphics();
g.drawImage(image, 0, 0, newWidth, newHeight, null);
g.dispose();
return scaled;
}
BinaryBitmap bitmap = new BinaryBitmap(new HybridBinarizer(source));
/**
* Quick check whether an image appears to be blank (single solid colour). Samples pixels at
* evenly-spaced positions — if all samples match the first pixel the image is almost certainly
* blank (e.g. a masked image that returned solid white).
*/
private static boolean isBlankImage(int[] pixels) {
if (pixels.length == 0) return true;
int first = pixels[0];
int step = Math.max(1, pixels.length / BLANK_CHECK_SAMPLES);
for (int i = step; i < pixels.length; i += step) {
if (pixels[i] != first) {
return false;
}
}
return true;
}
/**
* Try to decode a QR code from pre-extracted RGB pixel data using multiple binarization
* strategies. Returns the decoded text or null.
*
* <p>Strategy 1: HybridBinarizer — good for variable brightness (digital PDFs).
*
* <p>Strategy 2: GlobalHistogramBinarizer — better for scanned/noisy images with uniform
* lighting, and for QR codes with embedded logos that confuse the hybrid approach.
*/
private static String tryDecodeQR(int[] pixels, int width, int height) {
RGBLuminanceSource source = new RGBLuminanceSource(width, height, pixels);
MultiFormatReader reader = new MultiFormatReader();
// Strategy 1: HybridBinarizer — good for variable brightness (digital PDFs)
try {
Result result = new MultiFormatReader().decode(bitmap);
BinaryBitmap bitmap = new BinaryBitmap(new HybridBinarizer(source));
Result result = reader.decode(bitmap, DECODE_HINTS);
log.debug("QR detected via HybridBinarizer: '{}'", result.getText());
return result.getText();
} catch (NotFoundException e) {
return null; // there is no QR code in the image
// continue
}
// Strategy 2: GlobalHistogramBinarizer — better for scanned/noisy images
try {
BinaryBitmap bitmap = new BinaryBitmap(new GlobalHistogramBinarizer(source));
Result result = reader.decode(bitmap, DECODE_HINTS);
log.debug("QR detected via GlobalHistogramBinarizer: '{}'", result.getText());
return result.getText();
} catch (NotFoundException e) {
return null;
}
}
/**
* Attempt to decode a QR code from a BufferedImage. Handles downscaling for oversized images
* and skips blank images early.
*/
private static String decodeQRCode(BufferedImage bufferedImage) {
bufferedImage = downscaleIfNeeded(bufferedImage);
int width = bufferedImage.getWidth();
int height = bufferedImage.getHeight();
int[] pixels = new int[width * height];
bufferedImage.getRGB(0, 0, width, height, pixels, 0, width);
// Skip blank images early (e.g. masked images that decode to solid white)
if (isBlankImage(pixels)) {
log.debug("Skipping blank {}x{} image", width, height);
return null;
}
return tryDecodeQR(pixels, width, height);
}
/** Count the number of images embedded in a page's resources. */
private static int countPageImages(PDPage page) {
if (page.getResources() == null || page.getResources().getXObjectNames() == null) {
return 0;
}
int count = 0;
for (COSName name : page.getResources().getXObjectNames()) {
if (page.getResources().isImageXObject(name)) {
count++;
}
}
return count;
}
/**
* Extract images directly from a page's resources and check each for a QR code. Returns the QR
* code text if found, null otherwise.
*/
private static String checkPageImagesDirect(PDPage page) throws IOException {
if (page.getResources() == null || page.getResources().getXObjectNames() == null) {
return null;
}
for (COSName name : page.getResources().getXObjectNames()) {
if (!page.getResources().isImageXObject(name)) {
continue;
}
PDImageXObject imageObject = (PDImageXObject) page.getResources().getXObject(name);
BufferedImage image;
try {
image = imageObject.getImage();
} catch (OutOfMemoryError e) {
log.warn(
"Skipping oversized embedded image '{}' ({}x{}) - out of memory",
name.getName(),
imageObject.getWidth(),
imageObject.getHeight());
continue;
}
String result = decodeQRCode(image);
if (result != null) {
return result;
}
}
return null;
}
/**
* Render the full page to an image and scan it for a QR code. Tries a low DPI first (fast, low
* memory) and only retries at the system's maxDPI if detection fails. The first rendered image
* is released before the retry to allow GC to reclaim it.
*/
private String checkPageByRendering(PDFRenderer pdfRenderer, int pageNum) throws IOException {
log.debug("Rendering page {} at {} DPI for QR detection", pageNum + 1, QR_DETECTION_DPI);
BufferedImage bim =
ExceptionUtils.handleOomRendering(
pageNum + 1,
QR_DETECTION_DPI,
() -> pdfRenderer.renderImageWithDPI(pageNum, QR_DETECTION_DPI));
String result = decodeQRCode(bim);
bim = null; // allow GC before potential high-DPI retry
if (result == null) {
int maxDpi = getSystemMaxDpi();
if (maxDpi > QR_DETECTION_DPI) {
log.debug(
"Retrying page {} at {} DPI (low-DPI detection failed)",
pageNum + 1,
maxDpi);
BufferedImage highRes =
ExceptionUtils.handleOomRendering(
pageNum + 1,
maxDpi,
() -> pdfRenderer.renderImageWithDPI(pageNum, maxDpi));
result = decodeQRCode(highRes);
}
}
return result;
}
private int getSystemMaxDpi() {
if (applicationProperties != null && applicationProperties.getSystem() != null) {
return applicationProperties.getSystem().getMaxDPI();
}
return QR_DETECTION_DPI;
}
@AutoJobPostMapping(value = "/auto-split-pdf", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
@@ -111,42 +273,56 @@ public class AutoSplitPdfController {
summary = "Auto split PDF pages into separate documents",
description =
"This endpoint accepts a PDF file, scans each page for a specific QR code, and"
+ " splits the document at the QR code boundaries. The output is a zip file"
+ " containing each separate PDF document. Input:PDF Output:ZIP-PDF"
+ " splits the document at the QR code boundaries. The output is a zip"
+ " file containing each separate PDF document. Input:PDF Output:ZIP-PDF"
+ " Type:SISO")
public ResponseEntity<byte[]> autoSplitPdf(@ModelAttribute AutoSplitPdfRequest request)
throws IOException {
public ResponseEntity<StreamingResponseBody> autoSplitPdf(
@ModelAttribute AutoSplitPdfRequest request) throws IOException {
MultipartFile file = request.getFileInput();
boolean duplexMode = Boolean.TRUE.equals(request.getDuplexMode());
log.info(
"Auto-split starting: filename='{}', size={} bytes, duplexMode={}",
file.getOriginalFilename(),
file.getSize(),
duplexMode);
List<PDDocument> splitDocuments = new ArrayList<>();
try (TempFile outputTempFile = new TempFile(tempFileManager, ".zip");
PDDocument document = pdfDocumentFactory.load(file.getInputStream())) {
TempFile outputTempFile = new TempFile(tempFileManager, ".zip");
try (PDDocument document = pdfDocumentFactory.load(file.getInputStream())) {
int totalPages = document.getNumberOfPages();
log.info("PDF loaded, totalPages={}", totalPages);
PDFRenderer pdfRenderer = new PDFRenderer(document);
pdfRenderer.setSubsamplingAllowed(true);
for (int page = 0; page < document.getNumberOfPages(); ++page) {
BufferedImage bim;
for (int page = 0; page < totalPages; ++page) {
PDPage pdPage = document.getPage(page);
int imageCount = countPageImages(pdPage);
// Use global maximum DPI setting, fallback to 300 if not set
int renderDpi = 150; // Default fallback
ApplicationProperties properties =
ApplicationContextProvider.getBean(ApplicationProperties.class);
if (properties != null && properties.getSystem() != null) {
renderDpi = properties.getSystem().getMaxDPI();
String qrResult;
if (imageCount > 0 && imageCount <= MAX_IMAGES_FOR_DIRECT_EXTRACTION) {
// Try extracting images directly from the PDF (faster, avoids rendering)
qrResult = checkPageImagesDirect(pdPage);
if (qrResult == null) {
// Fall back to rendering — the image may use masking/compositing
// that getImage() doesn't resolve, or the QR may be vector-drawn
qrResult = checkPageByRendering(pdfRenderer, page);
}
} else {
// Too many images or no images — render the full page
qrResult = checkPageByRendering(pdfRenderer, page);
}
final int dpi = renderDpi;
final int pageNum = page;
bim =
ExceptionUtils.handleOomRendering(
pageNum + 1,
dpi,
() -> pdfRenderer.renderImageWithDPI(pageNum, dpi));
String result = decodeQRCode(bim);
boolean isValidQrCode = qrResult != null && VALID_QR_CONTENTS.contains(qrResult);
if (isValidQrCode) {
log.info(
"Page {}/{} contains QR divider ('{}')",
page + 1,
totalPages,
qrResult);
}
boolean isValidQrCode = VALID_QR_CONTENTS.contains(result);
log.debug("detected qr code {}, code is vale={}", result, isValidQrCode);
if (isValidQrCode && page != 0) {
splitDocuments.add(new PDDocument());
}
@@ -159,45 +335,36 @@ public class AutoSplitPdfController {
splitDocuments.add(firstDocument);
}
// If duplexMode is true and current page is a divider, then skip next page
if (duplexMode && isValidQrCode) {
page++;
page++; // skip back of divider page
}
}
// Remove split documents that have no pages
splitDocuments.removeIf(pdDocument -> pdDocument.getNumberOfPages() == 0);
log.info("Split complete, {} output documents", splitDocuments.size());
String filename =
GeneralUtils.removeExtension(
Filenames.toSimpleFileName(file.getOriginalFilename()));
try (ZipOutputStream zipOut =
new ZipOutputStream(Files.newOutputStream(outputTempFile.getPath()))) {
// Stream split documents directly into zip — avoids holding all PDFs in memory
try (OutputStream fileOut = Files.newOutputStream(outputTempFile.getPath());
ZipOutputStream zipOut = new ZipOutputStream(fileOut)) {
for (int i = 0; i < splitDocuments.size(); i++) {
String fileName = filename + "_" + (i + 1) + ".pdf";
PDDocument splitDocument = splitDocuments.get(i);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
splitDocument.save(baos);
byte[] pdf = baos.toByteArray();
ZipEntry pdfEntry = new ZipEntry(fileName);
zipOut.putNextEntry(pdfEntry);
zipOut.write(pdf);
zipOut.putNextEntry(new ZipEntry(fileName));
splitDocuments.get(i).save(zipOut);
zipOut.closeEntry();
}
}
byte[] data = Files.readAllBytes(outputTempFile.getPath());
return WebResponseUtils.bytesToWebResponse(
data, filename + ".zip", MediaType.APPLICATION_OCTET_STREAM);
return WebResponseUtils.zipFileToWebResponse(outputTempFile, filename + ".zip");
} catch (Exception e) {
outputTempFile.close();
log.error("Error in auto split", e);
throw e;
} finally {
// Clean up split documents
for (PDDocument splitDoc : splitDocuments) {
try {
splitDoc.close();
@@ -1,8 +1,9 @@
package stirling.software.SPDF.controller.api.misc;
import java.awt.image.BufferedImage;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.nio.file.Files;
import java.util.ArrayList;
import java.util.List;
import java.util.Locale;
@@ -19,6 +20,7 @@ import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import io.github.pixee.security.Filenames;
import io.swagger.v3.oas.annotations.Operation;
@@ -35,6 +37,8 @@ import stirling.software.common.util.ApplicationContextProvider;
import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.PdfUtils;
import stirling.software.common.util.TempFile;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
@MiscApi
@@ -43,6 +47,7 @@ import stirling.software.common.util.WebResponseUtils;
public class BlankPageController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final TempFileManager tempFileManager;
public static boolean isBlankImage(
BufferedImage image, int threshold, double whitePercent, int blurSize) {
@@ -83,7 +88,8 @@ public class BlankPageController {
"This endpoint removes blank pages from a given PDF file. Users can specify the"
+ " threshold and white percentage to tune the detection of blank pages."
+ " Input:PDF Output:PDF Type:SISO")
public ResponseEntity<byte[]> removeBlankPages(@ModelAttribute RemoveBlankPagesRequest request)
public ResponseEntity<StreamingResponseBody> removeBlankPages(
@ModelAttribute RemoveBlankPagesRequest request)
throws IOException, InterruptedException {
MultipartFile inputFile = request.getFileInput();
int threshold = request.getThreshold();
@@ -149,28 +155,29 @@ public class BlankPageController {
pageIndex++;
}
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ZipOutputStream zos = new ZipOutputStream(baos);
String filename =
GeneralUtils.removeExtension(
Filenames.toSimpleFileName(inputFile.getOriginalFilename()));
if (!nonBlankPages.isEmpty()) {
createZipEntry(zos, nonBlankPages, filename + "_nonBlankPages.pdf");
} else {
createZipEntry(zos, blankPages, filename + "_allBlankPages.pdf");
}
TempFile tempOut = tempFileManager.createManagedTempFile(".zip");
try (OutputStream fos = Files.newOutputStream(tempOut.getFile().toPath());
ZipOutputStream zos = new ZipOutputStream(fos)) {
if (!nonBlankPages.isEmpty()) {
createZipEntry(zos, nonBlankPages, filename + "_nonBlankPages.pdf");
} else {
createZipEntry(zos, blankPages, filename + "_allBlankPages.pdf");
}
if (!nonBlankPages.isEmpty() && !blankPages.isEmpty()) {
createZipEntry(zos, blankPages, filename + "_blankPages.pdf");
if (!nonBlankPages.isEmpty() && !blankPages.isEmpty()) {
createZipEntry(zos, blankPages, filename + "_blankPages.pdf");
}
} catch (IOException e) {
tempOut.close();
throw e;
}
zos.close();
log.info("Returning ZIP file: {}", filename + "_processed.zip");
return WebResponseUtils.baosToWebResponse(
baos, filename + "_processed.zip", MediaType.APPLICATION_OCTET_STREAM);
return WebResponseUtils.zipFileToWebResponse(tempOut, filename + "_processed.zip");
} catch (ExceptionUtils.OutOfMemoryDpiException e) {
throw e;
@@ -38,6 +38,7 @@ import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.server.ResponseStatusException;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import io.swagger.v3.oas.annotations.Operation;
@@ -328,7 +329,8 @@ public class CompressController {
+ "_"
+ image.getBitsPerComponent();
return bytesToHexString(generateMD5(enhancedData.getBytes()));
return bytesToHexString(
generateMD5(enhancedData.getBytes(StandardCharsets.UTF_8)));
}
return "empty-stream";
}
@@ -727,7 +729,8 @@ public class CompressController {
params.append("_").append(image.getDecode().toString());
}
return bytesToHexString(generateMD5(params.toString().getBytes()));
return bytesToHexString(
generateMD5(params.toString().getBytes(StandardCharsets.UTF_8)));
} catch (Exception e) {
return "fallback-decode-" + System.identityHashCode(image);
}
@@ -798,7 +801,8 @@ public class CompressController {
metadata.append("_softmask");
}
return bytesToHexString(generateMD5(metadata.toString().getBytes()));
return bytesToHexString(
generateMD5(metadata.toString().getBytes(StandardCharsets.UTF_8)));
} catch (Exception e) {
return "fallback-meta-" + System.identityHashCode(image);
}
@@ -924,8 +928,8 @@ public class CompressController {
description =
"This endpoint accepts a PDF file and optimizes it based on the provided"
+ " parameters. Input:PDF Output:PDF Type:SISO")
public ResponseEntity<byte[]> optimizePdf(@ModelAttribute OptimizePdfRequest request)
throws Exception {
public ResponseEntity<StreamingResponseBody> optimizePdf(
@ModelAttribute OptimizePdfRequest request) throws Exception {
MultipartFile inputFile = request.getFileInput();
// Validate input file
@@ -1097,7 +1101,8 @@ public class CompressController {
try {
try (PDDocument document = pdfDocumentFactory.load(currentFile.toFile())) {
return WebResponseUtils.pdfDocToWebResponse(document, outputFilename);
return WebResponseUtils.pdfDocToWebResponse(
document, outputFilename, tempFileManager);
}
} catch (IOException e) {
throw ExceptionUtils.handlePdfException(e, "PDF optimization");
@@ -23,6 +23,7 @@ import stirling.software.common.configuration.AppConfig;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.service.ServerCertificateServiceInterface;
import stirling.software.common.service.UserServiceInterface;
import stirling.software.common.util.GeneralUtils;
@ConfigApi
@Hidden
@@ -122,8 +123,17 @@ public class ConfigController {
configData.put("contextPath", appConfig.getContextPath());
configData.put("serverPort", appConfig.getServerPort());
// Add frontendUrl for mobile scanner QR codes
String frontendUrl = applicationProperties.getSystem().getFrontendUrl();
if ((frontendUrl == null || frontendUrl.isBlank())
&& Boolean.parseBoolean(
System.getProperty("STIRLING_PDF_TAURI_MODE", "false"))) {
String localIp = GeneralUtils.getLocalNetworkIp();
if (localIp != null) {
String scheme =
appConfig.getBackendUrl().startsWith("https") ? "https" : "http";
frontendUrl = scheme + "://" + localIp + ":" + appConfig.getServerPort();
}
}
configData.put("frontendUrl", frontendUrl != null ? frontendUrl : "");
// Add mobile scanner settings
@@ -1,6 +1,5 @@
package stirling.software.SPDF.controller.api.misc;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.util.HashSet;
@@ -14,6 +13,7 @@ import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import io.swagger.v3.oas.annotations.Operation;
@@ -26,6 +26,8 @@ import stirling.software.common.model.api.PDFFile;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.TempFile;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
@MiscApi
@@ -34,12 +36,13 @@ import stirling.software.common.util.WebResponseUtils;
public class DecompressPdfController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final TempFileManager tempFileManager;
@AutoJobPostMapping(value = "/decompress-pdf", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
@Operation(
summary = "Decompress PDF streams",
description = "Fully decompresses all PDF streams including text content")
public ResponseEntity<byte[]> decompressPdf(@ModelAttribute PDFFile request)
public ResponseEntity<StreamingResponseBody> decompressPdf(@ModelAttribute PDFFile request)
throws IOException {
MultipartFile file = request.getFileInput();
@@ -48,13 +51,18 @@ public class DecompressPdfController {
// Process all objects in document
processAllObjects(document);
// Save with explicit no compression
ByteArrayOutputStream baos = new ByteArrayOutputStream();
document.save(baos, CompressParameters.NO_COMPRESSION);
// Save with explicit no compression to a temp file
TempFile tempOut = tempFileManager.createManagedTempFile(".pdf");
try {
document.save(tempOut.getFile(), CompressParameters.NO_COMPRESSION);
} catch (IOException e) {
tempOut.close();
throw e;
}
// Return the PDF as a response
return WebResponseUtils.bytesToWebResponse(
baos.toByteArray(),
// Return the PDF as a streaming response
return WebResponseUtils.pdfFileToWebResponse(
tempOut,
GeneralUtils.generateFilename(file.getOriginalFilename(), "_decompressed.pdf"));
}
}
@@ -1,8 +1,8 @@
package stirling.software.SPDF.controller.api.misc;
import java.awt.image.BufferedImage;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.ArrayList;
@@ -21,6 +21,7 @@ import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import io.swagger.v3.oas.annotations.Operation;
@@ -39,6 +40,8 @@ import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.ProcessExecutor;
import stirling.software.common.util.ProcessExecutor.ProcessExecutorResult;
import stirling.software.common.util.TempFile;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
@MiscApi
@@ -49,6 +52,7 @@ public class ExtractImageScansController {
private static final String REPLACEFIRST = "[.][^.]+$";
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final TempFileManager tempFileManager;
@AutoJobPostMapping(
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
@@ -61,7 +65,7 @@ public class ExtractImageScansController {
+ " parameters. Users can specify angle threshold, tolerance, minimum area,"
+ " minimum contour area, and border size. Input:PDF Output:IMAGE/ZIP"
+ " Type:SIMO")
public ResponseEntity<byte[]> extractImageScans(
public ResponseEntity<StreamingResponseBody> extractImageScans(
@ModelAttribute ExtractImageScansRequest request)
throws IOException, InterruptedException {
MultipartFile inputFile = request.getFileInput();
@@ -71,9 +75,8 @@ public class ExtractImageScansController {
List<String> images = new ArrayList<>();
List<Path> tempImageFiles = new ArrayList<>();
Path tempInputFile;
Path tempZipFile = null;
List<TempFile> tempImageFiles = new ArrayList<>();
TempFile tempInputFile = null;
List<Path> tempDirs = new ArrayList<>();
if (!CheckProgramInstall.isPythonAvailable()) {
@@ -83,6 +86,8 @@ public class ExtractImageScansController {
String pythonVersion = CheckProgramInstall.getAvailablePythonCommand();
Path splitPhotosScript = GeneralUtils.extractScript("split_photos.py");
TempFile finalOutput = null;
boolean finalOutputOwnershipTransferred = false;
try {
// Check if input file is a PDF
if ("pdf".equalsIgnoreCase(extension)) {
@@ -96,7 +101,8 @@ public class ExtractImageScansController {
// Create images of all pages
for (int i = 0; i < pageCount; i++) {
// Create temp file to save the image
Path tempFile = Files.createTempFile("image_", ".png");
TempFile tempImage = tempFileManager.createManagedTempFile(".png");
tempImageFiles.add(tempImage);
// Render image and save as temp file
BufferedImage image;
@@ -116,18 +122,17 @@ public class ExtractImageScansController {
pageIndex + 1,
dpi,
() -> pdfRenderer.renderImageWithDPI(pageIndex, dpi));
ImageIO.write(image, "png", tempFile.toFile());
ImageIO.write(image, "png", tempImage.getFile());
// Add temp file path to images list
images.add(tempFile.toString());
tempImageFiles.add(tempFile);
images.add(tempImage.getAbsolutePath());
}
}
} else {
tempInputFile = Files.createTempFile("input_", "." + extension);
inputFile.transferTo(tempInputFile);
tempInputFile = tempFileManager.createManagedTempFile("." + extension);
inputFile.transferTo(tempInputFile.getFile());
// Add input file path to images list
images.add(tempInputFile.toString());
images.add(tempInputFile.getAbsolutePath());
}
List<byte[]> processedImageBytes = new ArrayList<>();
@@ -177,10 +182,10 @@ public class ExtractImageScansController {
if (processedImageBytes.size() > 1) {
String outputZipFilename =
GeneralUtils.generateFilename(fileName, "_processed.zip");
tempZipFile = Files.createTempFile("output_", ".zip");
finalOutput = tempFileManager.createManagedTempFile(".zip");
try (ZipOutputStream zipOut =
new ZipOutputStream(new FileOutputStream(tempZipFile.toFile()))) {
new ZipOutputStream(Files.newOutputStream(finalOutput.getPath()))) {
// Add processed images to the zip
for (int i = 0; i < processedImageBytes.size(); i++) {
ZipEntry entry =
@@ -193,13 +198,10 @@ public class ExtractImageScansController {
}
}
byte[] zipBytes = Files.readAllBytes(tempZipFile);
// Clean up the temporary zip file
Files.deleteIfExists(tempZipFile);
return WebResponseUtils.bytesToWebResponse(
zipBytes, outputZipFilename, MediaType.APPLICATION_OCTET_STREAM);
ResponseEntity<StreamingResponseBody> response =
WebResponseUtils.zipFileToWebResponse(finalOutput, outputZipFilename);
finalOutputOwnershipTransferred = true;
return response;
}
if (processedImageBytes.isEmpty()) {
throw ExceptionUtils.createIllegalArgumentException(
@@ -208,28 +210,28 @@ public class ExtractImageScansController {
// Return the processed image as a response
byte[] imageBytes = processedImageBytes.get(0);
return WebResponseUtils.bytesToWebResponse(
imageBytes,
GeneralUtils.generateFilename(fileName, ".png"),
MediaType.IMAGE_PNG);
finalOutput = tempFileManager.createManagedTempFile(".png");
try (OutputStream out = Files.newOutputStream(finalOutput.getPath())) {
out.write(imageBytes);
}
ResponseEntity<StreamingResponseBody> response =
WebResponseUtils.fileToWebResponse(
finalOutput,
GeneralUtils.generateFilename(fileName, ".png"),
MediaType.IMAGE_PNG);
finalOutputOwnershipTransferred = true;
return response;
}
} finally {
if (finalOutput != null && !finalOutputOwnershipTransferred) {
finalOutput.close();
}
// Cleanup logic for all temporary files and directories
tempImageFiles.forEach(
path -> {
try {
Files.deleteIfExists(path);
} catch (IOException e) {
log.error("Failed to delete temporary image file: {}", path, e);
}
});
tempImageFiles.forEach(TempFile::close);
if (tempZipFile != null && Files.exists(tempZipFile)) {
try {
Files.deleteIfExists(tempZipFile);
} catch (IOException e) {
log.error("Failed to delete temporary zip file: {}", tempZipFile, e);
}
if (tempInputFile != null) {
tempInputFile.close();
}
tempDirs.forEach(

Some files were not shown because too many files have changed in this diff Show More