Stirling-Tools/Stirling-PDF
1
0
Fork 0
mirror of https://github.com/Stirling-Tools/Stirling-PDF.git synced 2026-09-03 05:10:16 +03:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'main' into feat/auto-form-detection

Browse Source
This commit is contained in:
Frooodle
2026-07-29 17:41:29 +01:00
parent 7164bff8e3 8a5470dd01
commit d4275ff2cc
1552 changed files with 106825 additions and 25663 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.claude/skills/pr-quiz/SKILL.md
+137
View File
@@ -0,0 +1,137 @@
---
name: pr-quiz
description: >-
Quiz the PR author on their own branch before they request review, to prove they
actually understand the change - especially code an AI wrote for them. Scopes the
branch diff vs its base, reads the changed code, then asks graded questions about
what changed, why, how it works, what it could break, and which edge cases it must
handle. Presents all questions first, waits for the author's answers, then grades
each honestly against the real code (Correct / Partial / Incorrect with the true
answer and file:line), scores it, and gives a readiness verdict that names the
areas to re-study before asking humans to review. Use when asked to quiz me on my
PR/branch, "test my understanding before review", a self-check gate before opening
a PR, or before requesting reviewers. Administered as an interactive
multiple-choice quiz (clickable options) by default; pass --free-text for
written answers, --questions N to set count, --save to write a scorecard.
argument-hint: "[branch-or-base-ref] [--questions N] [--free-text] [--save]"
allowed-tools: Bash, Read, Grep, Glob, Write, AskUserQuestion
---
# PR Quiz
Test whether the **author** genuinely understands their own branch before they ask
other people to spend time reviewing it. This is a self-check gate: the point is to
catch changes - often AI-written - that the author would not be able to explain or
defend in review. Be a fair but honest examiner, not a pushover.
`$ARGUMENTS` may name a base ref or branch to diff against; default is this branch
vs where it forked from the main line. Flags:
- `--questions N` - target N questions (else scale to diff size, see below).
- `--free-text` - administer as a written numbered list instead of the default
interactive multiple-choice.
- `--save` - also write a scorecard file after grading.
## Integrity rules (read first - the whole skill depends on these)
1. **Present every question before revealing any answer.** Ask, then wait. Never
show the answer key alongside the questions.
2. **Do not give hints or the answer while the quiz is open.** If the author asks
"what's the answer?" or "is it X?" before committing, decline warmly and tell
them to give their best answer first - guessing is part of the signal.
3. **Grade truthfully.** Vague, hand-wavy, or "the AI did it" non-answers are
Partial or Incorrect, not Correct. Do not inflate the score to be nice; a false
pass defeats the entire purpose.
4. **Ground everything in code you actually read.** Every question and every model
answer must trace to a real line in the diff. Cite `path:line`. No trivia
("how many lines?"), no invented behavior.
5. **Credit real understanding.** If the author explains it correctly in their own
words, mark it Correct even if worded differently than your key.
## Process
### 1. Scope the change (silently)
- Find the base. Prefer the fork point off the main line so the quiz covers only
this branch's work:
```bash
git fetch -q origin 2>/dev/null; \
BASE=$(git merge-base HEAD origin/main 2>/dev/null || git merge-base HEAD main); \
git diff --stat "$BASE"...HEAD
```
If `$ARGUMENTS` names a ref, diff against that instead.
- If the diff is empty, stop and say there's nothing to quiz on.
- Read commit messages / PR description for the *stated* intent, but verify it
against the actual diff - a mismatch is itself a good question.
### 2. Understand the code well enough to examine on it
Read the full diff plus enough surrounding context and related files to answer
every question you plan to ask. You cannot grade understanding you don't have.
Note the non-obvious parts: the design decisions, the risky lines, the edge cases,
the cross-file ripples, and anything that violates or upholds repo conventions
(for this repo e.g. `@app/*` import layering, all file ops via FileContext,
Jackson 3 / Spring Boot 4 APIs, engine typed-contract boundaries).
### 3. Build the question set
Scale count to the change unless `--questions N` is given:
small (< ~50 changed lines) 3-4, medium 5-8, large 9-12. Cap at 12.
Draw from these categories - weight toward the ones the diff actually exercises:
- **Intent** - what problem this solves; why it was needed now.
- **Mechanism** - how a specific non-trivial piece actually works ("walk me
through what `foo()` does when called with X").
- **Decisions & alternatives** - why this approach over an obvious alternative;
what a reviewer would reasonably push back on.
- **Blast radius** - what else this touches or could break; what you'd retest.
- **Edge cases** - inputs/states the change must handle (null, empty, large,
concurrent, error paths).
- **Conventions & correctness** - does it follow the repo's rules; is there a
latent bug the author should be able to spot.
Prefer questions the author can only answer if they read and understood the code.
Keep a private answer key with `path:line` for each - do **not** show it yet.
### 4. Administer the quiz
- **Default (multiple choice):** use the `AskUserQuestion` tool. Per question write
3-4 options where **every** option is independently plausible - each distractor a
real-but-wrong reading of the code, not filler. Two hard rules so the answer
can't be spotted by shape rather than knowledge:
- **Randomise the correct option's position** across questions - never default
it to first. Spread it roughly evenly over the slots.
- **Keep all options the same depth and length.** Do not describe the correct
one more fully than the distractors - a longer or more-detailed option is a
dead giveaway. Trim the right answer or flesh out the wrong ones until a
reader can't tell them apart by size.
The tool caps a call at 4 questions, so ask in batches of 4 - but run them as
one continuous flow: fire the next batch immediately after the previous
returns, with no narration ("Round 2 of 3") and no grading between batches.
The author always has an "Other" free-text escape, which is fine.
- **`--free-text`:** present all questions in one numbered list, then say
"Answer in one reply; number your answers. I won't grade until you're done."
Wait for the author's answers.
- Do not proceed to grading until every answer is in.
### 5. Grade
For each question, in order:
- Verdict: **Correct** / **Partial** / **Incorrect**.
- The model answer in one or two sentences, citing the real `path:line`.
- One line on the gap when Partial/Incorrect - what they missed and where to look.
Then a **Score** (e.g. 6/8, counting Partial as half) and a one-line summary of
the pattern (e.g. "solid on intent, shaky on the error paths").
### 6. Readiness verdict
End with a clear call:
- **Ready for review** - understanding is sound; note anything to mention to
reviewers proactively.
- **Study first** - list the specific files/concepts to re-read before requesting
review, each as a clickable `path:line`. Be concrete: "re-read the null handling
in X before you send this out."
Keep it honest - if they'd get grilled in review on something, say so now.
### 7. If `--save`
Write `pr-quiz/<branch>-scorecard.md`: the questions, their answers, your grades
and model answers, the score, and the verdict. Don't commit it unless asked.
## Principles
- **The author is the examinee, not the collaborator.** During the quiz you withhold
answers; you're measuring them, not helping them pass.
- **A failed quiz is a successful outcome** - it caught a gap before a human's time
was spent. Frame it that way, not as a scolding.
- **True to the code.** Every question, answer, and grade traces to a line you read.
- **Terse and direct** in chat - the questions and the verdict, minimal preamble.
.github/aur/stirling-pdf-desktop/PKGBUILD
+1 -1
View File
@@ -1,6 +1,6 @@
# Maintainer: Stirling PDF Inc <contact@stirlingpdf.com>
pkgname=stirling-pdf-desktop
pkgver=2.14.1
pkgver=2.14.2
pkgrel=1
pkgdesc="Locally hosted, web-based PDF manipulation tool (Tauri desktop app, official Stirling PDF Inc build)"
arch=('x86_64')
.github/aur/stirling-pdf-server-bin/PKGBUILD
+1 -1
View File
@@ -1,6 +1,6 @@
# Maintainer: Stirling PDF Inc <contact@stirlingpdf.com>
pkgname=stirling-pdf-server-bin
pkgver=2.14.1
pkgver=2.14.2
pkgrel=1
pkgdesc="Locally hosted, web-based PDF manipulation tool (server JAR, prebuilt)"
arch=('any')
.github/config/.files.yaml
+32 -3
View File
@@ -1,16 +1,35 @@
# CI routing infra. Editing the top-level router (build.yml) or this filter
# config re-runs every area's jobs, so every job-gating filter below includes
# *ci. That makes a change to how jobs are dispatched actually exercise those
# jobs (self-testing), instead of a router edit only matching the project filter.
ci: &ci
- .github/workflows/build.yml
- .github/config/.files.yaml
build: &build
- *ci
- build.gradle
- gradle/spotless.gradle
- app/(common|core|proprietary|saas)/build.gradle
- Taskfile.yml
- .taskfiles/backend.yml
- .github/workflows/check-licence.yml
openapi: &openapi
- *ci
- *build
- app/(common|core|proprietary|saas)/src/main/java/**
- .github/workflows/check-openapi.yml
docker-base: &docker-base
- docker/base/Dockerfile
# Dockerfiles only (base + embedded + unoserver). Gates the slow multi-arch
# (arm64) leg of the PR docker test build: arm64 is only rebuilt when a
# Dockerfile itself changes, not on every code PR.
dockerfiles: &dockerfiles
- docker/**/Dockerfile*
docker: &docker
- docker/embedded/Dockerfile
- docker/embedded/Dockerfile.fat
@@ -23,13 +42,11 @@ docker: &docker
- *docker-base
project: &project
- *ci
- app/(common|core|proprietary|saas)/src/(main|test)/java/**
- *build
- "app/(common|core|proprietary|saas)/src/(main|test)/resources/**/!(messages_*.properties|*.md)*"
- exampleYmlFiles/**
- gradle/**
- libs/**
- "testing/**/!(requirements*.txt|requirements*.in)*"
- *docker
- *docker-base
- gradle.properties
@@ -45,8 +62,11 @@ project: &project
- .taskfiles/docker.yml
- scripts/db-migration/**
- .github/workflows/db-migration-test.yml
- .github/workflows/docker-compose-tests.yml
- .github/workflows/test-build-docker.yml
frontend: &frontend
- *ci
- frontend/**
- .github/workflows/testdriver.yml
- testing/**
@@ -63,10 +83,15 @@ frontend: &frontend
- Taskfile.yml
- .taskfiles/frontend.yml
- .taskfiles/e2e.yml
- .github/workflows/frontend-validation.yml
- .github/workflows/frontend-a11y.yml
- .github/workflows/e2e-stubbed.yml
- .github/workflows/e2e-live.yml
# Files that affect the Tauri desktop bundle. Gate the multi-OS Tauri build
# job on changes to any of these.
tauri: &tauri
- *ci
- frontend/editor/src-tauri/**
- frontend/editor/src/desktop/**
- frontend/editor/tsconfig.desktop.vite.json
@@ -81,6 +106,7 @@ tauri: &tauri
# the engine validation job on changes to engine sources or to the Java
# tool surfaces it generates models from.
engine: &engine
- *ci
- engine/**
- app/(common|core|proprietary|saas)/src/main/java/**
- .github/workflows/ai-engine.yml
@@ -93,6 +119,7 @@ engine: &engine
# tasks that drive generation. Deliberately excludes the broad frontend/docker/
# testing globs, so a CSS-only PR does not boot the backend to rebuild the spec.
generated-models: &generated-models
- *ci
- *openapi
- frontend/editor/scripts/generate-tool-api-types.mts
- frontend/editor/src/core/types/toolApiTypes.ts
@@ -115,6 +142,7 @@ licenses-backend: &licenses-backend
# Files that can affect premium / enterprise behaviour. Gate the enterprise
# Playwright job on changes to any of these on PRs.
proprietary: &proprietary
- *ci
- app/proprietary/**
- frontend/editor/src/proprietary/**
- frontend/editor/src/core/tests/enterprise/**
@@ -129,4 +157,5 @@ proprietary: &proprietary
- configs/settings.yml.template
- build.gradle
- app/proprietary/build.gradle
- gradle/spotless.gradle
- .github/workflows/build-enterprise.yml
.github/labeler-config-srvaroa.yml
+1 -1
View File
@@ -163,7 +163,7 @@ labels:
- '.github/workflows/scorecards.yml'
- 'exampleYmlFiles/test_cicd.yml'
- label: 'Github'
- label: 'GitHub'
files:
- '.github/.*'
.github/labels.yml
+11 -7
View File
@@ -5,6 +5,7 @@
# the GitHub Action https://github.com/marketplace/actions/github-labeler.
- name: "Licenses"
color: "EDEDED"
description: "Issues or pull requests related to licenses"
from_name: "licenses"
- name: "Back End"
color: "20CE6C"
@@ -146,21 +147,21 @@
description: "Changes that do not affect the meaning of the code (formatting, etc.)"
- name: "admin"
color: "195055"
- name: "codex"
color: "ededed"
description: null
- name: "Github"
- name: "GitHub"
color: "0052CC"
- name: "github_actions"
color: "000000"
description: "Pull requests that update GitHub Actions code"
description: "Issues or pull requests related to GitHub configuration and integrations"
from_name: "Github"
- name: "needs-changes"
color: "A65A86"
description: "Pull requests that require changes before they can be merged"
- name: "on-hold"
color: "2526F9"
- name: "python"
color: "2b67c6"
description: "Pull requests that update Python code"
- name: "engine"
color: "2b67c6"
description: "Issues or pull requests related to the engine"
- name: "size:L"
color: "eb9500"
description: "This PR changes 100-499 lines ignoring generated files."
@@ -201,3 +202,6 @@
- name: "license-review-required"
color: "EDEDED"
description: "This PR requires a license review"
- name: "has conflicts"
color: "D93F0B"
description: "Pull request has merge conflicts with the base branch"
.github/scripts/requirements_dev.txt
+365 -352
View File
@@ -1,101 +1,116 @@
#
# This file is autogenerated by pip-compile with Python 3.12
# This file is autogenerated by pip-compile with Python 3.13
# by the following command:
#
# pip-compile --allow-unsafe --generate-hashes --output-file='.github\scripts\requirements_dev.txt' --strip-extras '.github\scripts\requirements_dev.in'
#
# WARNING: pip install will require the following package to be hashed.
# Consider using a hashable URL like https://github.com/jazzband/pip-tools/archive/SOMECOMMIT.zip
# CVE-2025-6176 mitigation: pin brotli to a specific commit
brotli @ git+https://github.com/google/brotli.git@028fb5a23661f123017c060daa546b55cf4bde29
# via
# -r .github/scripts/requirements_dev.in
# fonttools
cffi==2.0.0 \
--hash=sha256:00bdf7acc5f795150faa6957054fbbca2439db2f775ce831222b66f192f03beb \
--hash=sha256:07b271772c100085dd28b74fa0cd81c8fb1a3ba18b21e03d7c27f3436a10606b \
--hash=sha256:087067fa8953339c723661eda6b54bc98c5625757ea62e95eb4898ad5e776e9f \
--hash=sha256:0a1527a803f0a659de1af2e1fd700213caba79377e27e4693648c2923da066f9 \
--hash=sha256:0cf2d91ecc3fcc0625c2c530fe004f82c110405f101548512cce44322fa8ac44 \
--hash=sha256:0f6084a0ea23d05d20c3edcda20c3d006f9b6f3fefeac38f59262e10cef47ee2 \
--hash=sha256:12873ca6cb9b0f0d3a0da705d6086fe911591737a59f28b7936bdfed27c0d47c \
--hash=sha256:19f705ada2530c1167abacb171925dd886168931e0a7b78f5bffcae5c6b5be75 \
--hash=sha256:1cd13c99ce269b3ed80b417dcd591415d3372bcac067009b6e0f59c7d4015e65 \
--hash=sha256:1e3a615586f05fc4065a8b22b8152f0c1b00cdbc60596d187c2a74f9e3036e4e \
--hash=sha256:1f72fb8906754ac8a2cc3f9f5aaa298070652a0ffae577e0ea9bd480dc3c931a \
--hash=sha256:1fc9ea04857caf665289b7a75923f2c6ed559b8298a1b8c49e59f7dd95c8481e \
--hash=sha256:203a48d1fb583fc7d78a4c6655692963b860a417c0528492a6bc21f1aaefab25 \
--hash=sha256:2081580ebb843f759b9f617314a24ed5738c51d2aee65d31e02f6f7a2b97707a \
--hash=sha256:21d1152871b019407d8ac3985f6775c079416c282e431a4da6afe7aefd2bccbe \
--hash=sha256:24b6f81f1983e6df8db3adc38562c83f7d4a0c36162885ec7f7b77c7dcbec97b \
--hash=sha256:256f80b80ca3853f90c21b23ee78cd008713787b1b1e93eae9f3d6a7134abd91 \
--hash=sha256:28a3a209b96630bca57cce802da70c266eb08c6e97e5afd61a75611ee6c64592 \
--hash=sha256:2c8f814d84194c9ea681642fd164267891702542f028a15fc97d4674b6206187 \
--hash=sha256:2de9a304e27f7596cd03d16f1b7c72219bd944e99cc52b84d0145aefb07cbd3c \
--hash=sha256:38100abb9d1b1435bc4cc340bb4489635dc2f0da7456590877030c9b3d40b0c1 \
--hash=sha256:3925dd22fa2b7699ed2617149842d2e6adde22b262fcbfada50e3d195e4b3a94 \
--hash=sha256:3e17ed538242334bf70832644a32a7aae3d83b57567f9fd60a26257e992b79ba \
--hash=sha256:3e837e369566884707ddaf85fc1744b47575005c0a229de3327f8f9a20f4efeb \
--hash=sha256:3f4d46d8b35698056ec29bca21546e1551a205058ae1a181d871e278b0b28165 \
--hash=sha256:44d1b5909021139fe36001ae048dbdde8214afa20200eda0f64c068cac5d5529 \
--hash=sha256:45d5e886156860dc35862657e1494b9bae8dfa63bf56796f2fb56e1679fc0bca \
--hash=sha256:4647afc2f90d1ddd33441e5b0e85b16b12ddec4fca55f0d9671fef036ecca27c \
--hash=sha256:4671d9dd5ec934cb9a73e7ee9676f9362aba54f7f34910956b84d727b0d73fb6 \
--hash=sha256:53f77cbe57044e88bbd5ed26ac1d0514d2acf0591dd6bb02a3ae37f76811b80c \
--hash=sha256:5eda85d6d1879e692d546a078b44251cdd08dd1cfb98dfb77b670c97cee49ea0 \
--hash=sha256:5fed36fccc0612a53f1d4d9a816b50a36702c28a2aa880cb8a122b3466638743 \
--hash=sha256:61d028e90346df14fedc3d1e5441df818d095f3b87d286825dfcbd6459b7ef63 \
--hash=sha256:66f011380d0e49ed280c789fbd08ff0d40968ee7b665575489afa95c98196ab5 \
--hash=sha256:6824f87845e3396029f3820c206e459ccc91760e8fa24422f8b0c3d1731cbec5 \
--hash=sha256:6c6c373cfc5c83a975506110d17457138c8c63016b563cc9ed6e056a82f13ce4 \
--hash=sha256:6d02d6655b0e54f54c4ef0b94eb6be0607b70853c45ce98bd278dc7de718be5d \
--hash=sha256:6d50360be4546678fc1b79ffe7a66265e28667840010348dd69a314145807a1b \
--hash=sha256:730cacb21e1bdff3ce90babf007d0a0917cc3e6492f336c2f0134101e0944f93 \
--hash=sha256:737fe7d37e1a1bffe70bd5754ea763a62a066dc5913ca57e957824b72a85e205 \
--hash=sha256:74a03b9698e198d47562765773b4a8309919089150a0bb17d829ad7b44b60d27 \
--hash=sha256:7553fb2090d71822f02c629afe6042c299edf91ba1bf94951165613553984512 \
--hash=sha256:7a66c7204d8869299919db4d5069a82f1561581af12b11b3c9f48c584eb8743d \
--hash=sha256:7cc09976e8b56f8cebd752f7113ad07752461f48a58cbba644139015ac24954c \
--hash=sha256:81afed14892743bbe14dacb9e36d9e0e504cd204e0b165062c488942b9718037 \
--hash=sha256:8941aaadaf67246224cee8c3803777eed332a19d909b47e29c9842ef1e79ac26 \
--hash=sha256:89472c9762729b5ae1ad974b777416bfda4ac5642423fa93bd57a09204712322 \
--hash=sha256:8ea985900c5c95ce9db1745f7933eeef5d314f0565b27625d9a10ec9881e1bfb \
--hash=sha256:8eca2a813c1cb7ad4fb74d368c2ffbbb4789d377ee5bb8df98373c2cc0dee76c \
--hash=sha256:92b68146a71df78564e4ef48af17551a5ddd142e5190cdf2c5624d0c3ff5b2e8 \
--hash=sha256:9332088d75dc3241c702d852d4671613136d90fa6881da7d770a483fd05248b4 \
--hash=sha256:94698a9c5f91f9d138526b48fe26a199609544591f859c870d477351dc7b2414 \
--hash=sha256:9a67fc9e8eb39039280526379fb3a70023d77caec1852002b4da7e8b270c4dd9 \
--hash=sha256:9de40a7b0323d889cf8d23d1ef214f565ab154443c42737dfe52ff82cf857664 \
--hash=sha256:a05d0c237b3349096d3981b727493e22147f934b20f6f125a3eba8f994bec4a9 \
--hash=sha256:afb8db5439b81cf9c9d0c80404b60c3cc9c3add93e114dcae767f1477cb53775 \
--hash=sha256:b18a3ed7d5b3bd8d9ef7a8cb226502c6bf8308df1525e1cc676c3680e7176739 \
--hash=sha256:b1e74d11748e7e98e2f426ab176d4ed720a64412b6a15054378afdb71e0f37dc \
--hash=sha256:b21e08af67b8a103c71a250401c78d5e0893beff75e28c53c98f4de42f774062 \
--hash=sha256:b4c854ef3adc177950a8dfc81a86f5115d2abd545751a304c5bcf2c2c7283cfe \
--hash=sha256:b882b3df248017dba09d6b16defe9b5c407fe32fc7c65a9c69798e6175601be9 \
--hash=sha256:baf5215e0ab74c16e2dd324e8ec067ef59e41125d3eade2b863d294fd5035c92 \
--hash=sha256:c649e3a33450ec82378822b3dad03cc228b8f5963c0c12fc3b1e0ab940f768a5 \
--hash=sha256:c654de545946e0db659b3400168c9ad31b5d29593291482c43e3564effbcee13 \
--hash=sha256:c6638687455baf640e37344fe26d37c404db8b80d037c3d29f58fe8d1c3b194d \
--hash=sha256:c8d3b5532fc71b7a77c09192b4a5a200ea992702734a2e9279a37f2478236f26 \
--hash=sha256:cb527a79772e5ef98fb1d700678fe031e353e765d1ca2d409c92263c6d43e09f \
--hash=sha256:cf364028c016c03078a23b503f02058f1814320a56ad535686f90565636a9495 \
--hash=sha256:d48a880098c96020b02d5a1f7d9251308510ce8858940e6fa99ece33f610838b \
--hash=sha256:d68b6cef7827e8641e8ef16f4494edda8b36104d79773a334beaa1e3521430f6 \
--hash=sha256:d9b29c1f0ae438d5ee9acb31cadee00a58c46cc9c0b2f9038c6b0b3470877a8c \
--hash=sha256:d9b97165e8aed9272a6bb17c01e3cc5871a594a446ebedc996e2397a1c1ea8ef \
--hash=sha256:da68248800ad6320861f129cd9c1bf96ca849a2771a59e0344e88681905916f5 \
--hash=sha256:da902562c3e9c550df360bfa53c035b2f241fed6d9aef119048073680ace4a18 \
--hash=sha256:dbd5c7a25a7cb98f5ca55d258b103a2054f859a46ae11aaf23134f9cc0d356ad \
--hash=sha256:dd4f05f54a52fb558f1ba9f528228066954fee3ebe629fc1660d874d040ae5a3 \
--hash=sha256:de8dad4425a6ca6e4e5e297b27b5c824ecc7581910bf9aee86cb6835e6812aa7 \
--hash=sha256:e11e82b744887154b182fd3e7e8512418446501191994dbf9c9fc1f32cc8efd5 \
--hash=sha256:e6e73b9e02893c764e7e8d5bb5ce277f1a009cd5243f8228f75f842bf937c534 \
--hash=sha256:f73b96c41e3b2adedc34a7356e64c8eb96e03a3782b535e043a986276ce12a49 \
--hash=sha256:f93fd8e5c8c0a4aa1f424d6173f14a892044054871c771f8566e4008eaa359d2 \
--hash=sha256:fc33c5141b55ed366cfaad382df24fe7dcbc686de5be719b207bb248e3053dc5 \
--hash=sha256:fc7de24befaeae77ba923797c7c87834c73648a05a4bde34b3b7e5588973a453 \
--hash=sha256:fe562eb1a64e67dd297ccc4f5addea2501664954f2692b69a76449ec7913ecbf
cffi==2.1.0 \
--hash=sha256:02cb7ff33ded4f1532476731f89ede53e2e488a8e6205515a82144246ffa7dcc \
--hash=sha256:03e9810d18c646077e501f661b682fbf5dee4676048527ca3cffe66faa9960dd \
--hash=sha256:0520e1f4c35f44e209cbbb421b67eec42e6a157f59444dfb6058874ff3610e5d \
--hash=sha256:0582a58f3051372229ca8e7f5f589f9e5632678208d8636fea3676711fdf7fe5 \
--hash=sha256:0611e7ebf90573a535ebdc33ae9da222d037853983e13359f580fab781ca017f \
--hash=sha256:0a42c688d19fca6e095a53c6a6e2295a5b050a8b289f109adab02a9e61a25de6 \
--hash=sha256:0a96b74cda968eebbad56d973efe5098974f0a9fb323865bf99ea1fd24e3e64c \
--hash=sha256:10537b1df4967ca26d21e5072d7d54188354483b91dc75058968d3f0cf13fbda \
--hash=sha256:11b3fb55f4f8ad92274ed26705f65d8f91457de71f5380061eb6d125a768fecd \
--hash=sha256:15faec4adfff450819f3aee0e2e02c812de6edb88203aa58807955db2003472a \
--hash=sha256:164bff1657b2a74f0b6d54e11c9b375bc97b931f2ca9c43fcf875838da1570dd \
--hash=sha256:1854b724d00f6654c742097d5387569021be12d3a0f770eae1df8f8acfcc6acd \
--hash=sha256:19c54ac121cad98450b4896fa9a43ee0180d57bc4bc911a33db6cab1efab6cd3 \
--hash=sha256:1b96bfe2c4bd825681b7d311ad6d9b7280a091f43e8f63da5729638083cd3bfb \
--hash=sha256:1e9f50d192a3e525b15a75ab5114e442d83d657b7ec29182a991bc9a88fd3a66 \
--hash=sha256:1ff3456eab0d889592d1936d6125bbfbc7ae4d3354a700f8bd80450a66445d4d \
--hash=sha256:2282cd5e38aa8accd03e99d1256af8411c84cdbee6a89d841b563fdbd1f3e50f \
--hash=sha256:276f20fffd7b396e12516ba8edf9509210ac248cbbc5acbc39cd512f9f59ebe6 \
--hash=sha256:2b71d409cccee78310ab5dec549aed052aaea483346e282c7b02362596e01bb0 \
--hash=sha256:2e9dabb9abcb7ad15938c7196ad5c1718a4e6d33cc79b4c0209bdb64c4a54a5c \
--hash=sha256:30b65779d598c370374fefabf138d456fd6f3216bfa7bedfab1ba82025b0cd93 \
--hash=sha256:33eb1ad83ebe8f313e0df035c406227d55a79456704a863fad9842136af5ad7d \
--hash=sha256:35aaea0c7ee0e58a5cd8c2fd1a48fdf7ece0d2699b7ecdda08194e9ce5dd9b3d \
--hash=sha256:3681e031db29958a7502f5c0c9d6bbc4c36cb20f7b104086fa642d1799631ff8 \
--hash=sha256:379de10ce1ba048b1448599d1b37b24caee16309d1ac98d3982fc997f768700b \
--hash=sha256:37f525a7e7e50c017fdebe58b787be310ad59357ae43a053943a6e1a6c526001 \
--hash=sha256:3b926723c13eba9f81d2ef3820d63aeceec3b2d4639906047bf675cb8a7a500d \
--hash=sha256:3d7f118b5adbfdfead90c25822690b02bc8074fba949bb7858bec4ebd55adb43 \
--hash=sha256:46b1c8db8f6122420f32d02fffb924c2fe9bc772d228c7c711748fff56aabb2b \
--hash=sha256:47ff3a8bfd8cb9da1af7524b965127095055654c177fcfc7578debcb015eecd0 \
--hash=sha256:4d433a51f1870e43a13b6732f92aaf540ff77c2015097c78556f75a2d6c030e0 \
--hash=sha256:4f26194e3d95e06501b942642855aed4f953d55e95d7d01b7c4483db3ecff458 \
--hash=sha256:510aeeeac94811b138077451da1fb18b308a5feab47dd2b603af55804155e1c8 \
--hash=sha256:5972433ad71a9e46516584ef60a0fda12d9dc459938d1539c3ddecf9bdc1368d \
--hash=sha256:5ecbd0499275d57506d397eebe1981cee87b47fcd9ef5c22cab7ed7644a39a94 \
--hash=sha256:6274dcb2d15cef48daa73ed1be5a40d501d74dccd0cd6db364776d12cb6ba022 \
--hash=sha256:63960549e4f8dc41e31accb97b975abaecfc44c03e396c093a6436763c2ea7db \
--hash=sha256:64c753a0f87a256020004f37a1c8c02c480e725f910f0b2a0f3f07debd1b2479 \
--hash=sha256:6af371f3767faeffc6ac1ef57cdfd25844403e9d3f476c5537caee499de96376 \
--hash=sha256:6ca4919c6e4f89aa99c42510b42cf54596892c00b3f9077f6bdd1505e24b9c8d \
--hash=sha256:6d194185eabd279f1c05ebe3504265ddfc5ad2b58d0714f7db9f01da592e9eb6 \
--hash=sha256:702c436735fbe99d59ada02a1f65cfc0d31c0ee8b7290912f8fbc5cd1e4b16c3 \
--hash=sha256:716ff8ec22f20b4d988b12884086bcef0fc99737043e503f7a3935a6be99b1ea \
--hash=sha256:762f99479dcb369f60ab9017ad4ab97a36a1dd7c1ee5a3b15db0f4b8659120cd \
--hash=sha256:7762faa47e8ff7eb80bd261d9a7d8eea2d8baa69de5e95b70c1f338bbe712f02 \
--hash=sha256:78474632761faa0fb96f30b1c928c84ebcf68713cbb80d15bab09dfe61640fde \
--hash=sha256:799416bae98336e400981ff6e532d67d5c709cfb30afb79865a1315f94b0e224 \
--hash=sha256:7d034dcffa09e9a46c93fa3a3be402096cb5354ac6e41ab8e5cc9cd8b642ad76 \
--hash=sha256:7d28dff1db6764108bc30788d85d61c876beff416d9a49cb9dd7c5a9f34f5804 \
--hash=sha256:7d3538f9c0e50670f4deb93dbb696576e60590369cae2faf7de681e597a8a1f1 \
--hash=sha256:7d5980a3433d4b71a5e120f9dd551403d7824e31e2e67124fe2769c404c06913 \
--hash=sha256:7ea6b3e2c4250ff1de21c630fe72d0f63eb95c2c32ffbf64a358cf4a8836d714 \
--hash=sha256:86cf8755a791f72c85dc287128cc62d4f24d392e3f1e15837245623f4a33cccc \
--hash=sha256:88023dfe18799507b73f1dbb0d14326a17465de1bc9c9c7655c22845e9ddc3a2 \
--hash=sha256:89095c1968b4ba8285840e131bf2891b09ae137fe2146905acae0354fbce1b5e \
--hash=sha256:8d35c139744adb3e727cd51b1a18324bbe44b8bd41bf8322bca4d41289f48eda \
--hash=sha256:8e74a6135550c4748af665b1b1118b6aab33b1fc6a16f9aff630af107c3b4512 \
--hash=sha256:8f9ec95b8a043d3dfbc74d9abc6f7baf524dd27a8dc160b0a32ff9cdab650c28 \
--hash=sha256:90bec57cf82089383bd06a605b3eb8daebf7e5a668520beaf6e327a83a947699 \
--hash=sha256:95f2954c2c9473d892eca6e0409f3568b37ab62a8eedb122461f73cc273476e3 \
--hash=sha256:961be50688f7fba2fa65f63712d3b9b341a22311f5253460ce933f52f0de1c8c \
--hash=sha256:98fff996e983a36d3aa2eca83af40c5821202e7e6f32d13ae94e3d2286f10cfe \
--hash=sha256:9b8f0f26ca4e7513c534d351eca551947d053fac438f2a04ac96d882909b0d3a \
--hash=sha256:9d72af0cf10a76a600a9690078fe31c63b9588c8e86bf9fd353f713c84b5db0f \
--hash=sha256:9d8272c0e483b024e1b9ad029821470ed8ec65631dbd90217469da0e7cd89f1c \
--hash=sha256:a016194dbe13d14ee9556e734b772d8d67b947092b268d757fd4290e3ba2dfc2 \
--hash=sha256:a5781494d4d400a3f47f8f1da94b324f6e6b440a53387774002890a2a2f4b50f \
--hash=sha256:a95b05f9baf29b91171b3a8bd2020b028835243e7b0ff6bb23e2a3c228518b1b \
--hash=sha256:aa7a1b53a2a4452ada2d1b5dade9960b2522f1e61293a811a077439e39029565 \
--hash=sha256:ac0f1a2d0cfa7eea3f2aaf006ab6e70e8feeb16b75d65b7e5939982ca2f11056 \
--hash=sha256:af5e2915d41fe6c961694d7bfdc8562942638200f3ce2765dfb8b745cf997629 \
--hash=sha256:b6422532152adf4e59b110cb2808cee7a033800952f5c036b4af047ee43199e7 \
--hash=sha256:b65f590ef2a44640f9a05dbb548a429b4ade77913ce683ac8b1480777658a6c0 \
--hash=sha256:ba00f661f8ba35d075c937174e27c2c421cec3942fd2e0ea3e66996757c0fdd9 \
--hash=sha256:bccbbb5ee76a61f9d99b5bf3846a51d7fca4b6a732fe46f89295610edaf41853 \
--hash=sha256:bf01d8c84cbea96b944c73b22182e6c7c432b3475632b8111dbfdc95ddad6e13 \
--hash=sha256:bf5c6cf48238b0eb4c086978c492ad1cbc22373fc5b2d7353b3a598ce6db887a \
--hash=sha256:c16914df9fb7f500e440e6875fa23ff5e0b31db01fa9c06af98d59a91f0dc2e4 \
--hash=sha256:c351efb95e832a853a29361675f33a7ce53de1a109cd73fd47af0712213aa4ce \
--hash=sha256:c4165821e131d6d4ca444347c2b694e2311bcfa3fe5a861cc72968f28867beac \
--hash=sha256:c5f5df567f6eb216de69be06ce55c8b714090fae02b18a3b40da8163b8c5fa9c \
--hash=sha256:c941bb58d5a6e1c3892d86e42927ed6c180302f07e6d395d08c416e594b98b46 \
--hash=sha256:c97f080ea627e2863524c5af3836e2270b5f5dfff1f104392b959f8df0c5d384 \
--hash=sha256:cb96698e3c7413d906ce83f8ffd245ec1bd94707541f299d0ce4d6b0193e982b \
--hash=sha256:cbb7640ce37159548d2147b5b8c241f962143d4c71231431820783f4dc78f210 \
--hash=sha256:cdf2448aab5f661c9315308ec8b93f4e8a1a67a3c733f8631067a2b67d5913dc \
--hash=sha256:d2117334c3af3bdcb9a88522b844a2bdb5efdc4f71c6c822df55486ae1c3347a \
--hash=sha256:d53d10f7da99ae46f7373b9150393e9c5eab9b224909982b43832668de4779f5 \
--hash=sha256:d9fafc5aa2e2a39aaf7f8cc0c1f044a9b07fca12e558dca53a3cc5c654ad67a7 \
--hash=sha256:db3eb7d46527159a878ec3460e9d40615bc25ba337d477db681aea6e4f05c5d2 \
--hash=sha256:dbf7c7a88e2bac086f06d14577332760bdeecc42bdec8ac4077f6260557d9326 \
--hash=sha256:df2b82571a1b30f58a87bf4e5a9e78d2b1eff6c6ce8fd3aa3757221f93f0863f \
--hash=sha256:df92f2aba50eb4d96718b68ef76f2e57a57b54f2fa62333496d16c6d585a85ca \
--hash=sha256:eb4e8997a49aa2c08a3e43c9045d224448b8941d88e7ac163c7d383e560cbf98 \
--hash=sha256:efc1cdd798b1aaf39b4610bba7aad28c9bea9b910f25c784ccf9ec1fa719d1f9 \
--hash=sha256:f146d154428a2523f9cc7936c02353c2459b8f6cf07d3cd1ee1c0a611109c5d5 \
--hash=sha256:f5bce581e6b8c235e566a14768a943b172ada3ed73537bb0c0be1edee312d4e7 \
--hash=sha256:f9912624a0c0b834b7520d7769b3644453aabc0a7e1c839da7359f050750e9bc \
--hash=sha256:fb62edb5bb52cca65fab91a63afa7561607120d26090a7e8fda6fb9f064726da \
--hash=sha256:ff067a8d8d880e7809e4ac88eb009bb848870115317b306666502ccad30b147f
# via weasyprint
cfgv==3.5.0 \
--hash=sha256:a8dc6b26ad22ff227d2634a65cb388215ce6cc96bbcc5cfde7641ae87e8dacc0 \
@@ -105,67 +120,67 @@ cssselect2==0.9.0 \
--hash=sha256:6a99e5f91f9a016a304dd929b0966ca464bcfda15177b6fb4a118fc0fb5d9563 \
--hash=sha256:759aa22c216326356f65e62e791d66160a0f9c91d1424e8d8adc5e74dddfc6fb
# via weasyprint
distlib==0.4.0 \
--hash=sha256:9659f7d87e46584a30b5780e43ac7a2143098441670ff0a49d5f9034c54a6c16 \
--hash=sha256:feec40075be03a04501a973d81f633735b4b69f98b05450592310c0f401a4e0d
distlib==0.4.3 \
--hash=sha256:4b0ce306c966eb73bc3a7b6abad017c556dadd92c44701562cd528ac7fde4d5b \
--hash=sha256:f152097224a0ae24be5a0f6bae1b9359af82133bce63f98a95f86cae1aede9ed
# via virtualenv
filelock==3.29.0 \
--hash=sha256:69974355e960702e789734cb4871f884ea6fe50bd8404051a3530bc07809cf90 \
--hash=sha256:96f5f6344709aa1572bbf631c640e4ebeeb519e08da902c39a001882f30ac258
filelock==3.30.0 \
--hash=sha256:1774e682dbe443bd60f9609162fc596e2c80dc84ffc2957068953406d0520090 \
--hash=sha256:40632998f0772e64183bb819f086a1b9def6be1090cf1dcb9d45f46806ef279b
# via
# python-discovery
# virtualenv
fonttools==4.62.1 \
--hash=sha256:0aa72c43a601cfa9273bb1ae0518f1acadc01ee181a6fc60cd758d7fdadffc04 \
--hash=sha256:0b3ae47e8636156a9accff64c02c0924cbebad62854c4a6dbdc110cd5b4b341a \
--hash=sha256:12859ff0b47dd20f110804c3e0d0970f7b832f561630cd879969011541a464a9 \
--hash=sha256:149f7d84afca659d1a97e39a4778794a2f83bf344c5ee5134e09995086cc2392 \
--hash=sha256:1596aeaddf7f78e21e68293c011316a25267b3effdaccaf4d59bc9159d681b82 \
--hash=sha256:19177c8d96c7c36359266e571c5173bcee9157b59cfc8cb0153c5673dc5a3a7d \
--hash=sha256:1c5c25671ce8805e0d080e2ffdeca7f1e86778c5cbfbeae86d7f866d8830517b \
--hash=sha256:1eecc128c86c552fb963fe846ca4e011b1be053728f798185a1687502f6d398e \
--hash=sha256:268abb1cb221e66c014acc234e872b7870d8b5d4657a83a8f4205094c32d2416 \
--hash=sha256:2d850f66830a27b0d498ee05adb13a3781637b1826982cd7e2b3789ef0cc71ae \
--hash=sha256:2e7abd2b1e11736f58c1de27819e1955a53267c21732e78243fa2fa2e5c1e069 \
--hash=sha256:403d28ce06ebfc547fbcb0cb8b7f7cc2f7a2d3e1a67ba9a34b14632df9e080f9 \
--hash=sha256:40975849bac44fb0b9253d77420c6d8b523ac4dcdcefeff6e4d706838a5b80f7 \
--hash=sha256:486f32c8047ccd05652aba17e4a8819a3a9d78570eb8a0e3b4503142947880ed \
--hash=sha256:49a445d2f544ce4a69338694cad575ba97b9a75fff02720da0882d1a73f12800 \
--hash=sha256:59b372b4f0e113d3746b88985f1c796e7bf830dd54b28374cd85c2b8acd7583e \
--hash=sha256:5a648bde915fba9da05ae98856987ca91ba832949a9e2888b48c47ef8b96c5a9 \
--hash=sha256:5f37df1cac61d906e7b836abe356bc2f34c99d4477467755c216b72aa3dc748b \
--hash=sha256:6706d1cb1d5e6251a97ad3c1b9347505c5615c112e66047abbef0f8545fa30d1 \
--hash=sha256:68959f5fc58ed4599b44aad161c2837477d7f35f5f79402d97439974faebfebe \
--hash=sha256:6acb4109f8bee00fec985c8c7afb02299e35e9c94b57287f3ea542f28bd0b0a7 \
--hash=sha256:7487782e2113861f4ddcc07c3436450659e3caa5e470b27dc2177cade2d8e7fd \
--hash=sha256:7aa21ff53e28a9c2157acbc44e5b401149d3c9178107130e82d74ceb500e5056 \
--hash=sha256:7bca7a1c1faf235ffe25d4f2e555246b4750220b38de8261d94ebc5ce8a23c23 \
--hash=sha256:8d337fdd49a79b0d51c4da87bc38169d21c3abbf0c1aa9367eff5c6656fb6dae \
--hash=sha256:8f8fca95d3bb3208f59626a4b0ea6e526ee51f5a8ad5d91821c165903e8d9260 \
--hash=sha256:90365821debbd7db678809c7491ca4acd1e0779b9624cdc6ddaf1f31992bf974 \
--hash=sha256:92bb00a947e666169c99b43753c4305fc95a890a60ef3aeb2a6963e07902cc87 \
--hash=sha256:93c316e0f5301b2adbe6a5f658634307c096fd5aae60a5b3412e4f3e1728ab24 \
--hash=sha256:942b03094d7edbb99bdf1ae7e9090898cad7bf9030b3d21f33d7072dbcb51a53 \
--hash=sha256:9c125ffa00c3d9003cdaaf7f2c79e6e535628093e14b5de1dccb08859b680936 \
--hash=sha256:9dde91633f77fa576879a0c76b1d89de373cae751a98ddf0109d54e173b40f14 \
--hash=sha256:9e7863e10b3de72376280b515d35b14f5eeed639d1aa7824f4cf06779ec65e42 \
--hash=sha256:a24decd24d60744ee8b4679d38e88b8303d86772053afc29b19d23bb8207803c \
--hash=sha256:a5d8825e1140f04e6c99bb7d37a9e31c172f3bc208afbe02175339e699c710e1 \
--hash=sha256:aa69d10ed420d8121118e628ad47d86e4caa79ba37f968597b958f6cceab7eca \
--hash=sha256:ad5cca75776cd453b1b035b530e943334957ae152a36a88a320e779d61fc980c \
--hash=sha256:b4e0fcf265ad26e487c56cb12a42dffe7162de708762db951e1b3f755319507d \
--hash=sha256:b820fcb92d4655513d8402d5b219f94481c4443d825b4372c75a2072aa4b357a \
--hash=sha256:bd13b7999d59c5eb1c2b442eb2d0c427cb517a0b7a1f5798fc5c9e003f5ff782 \
--hash=sha256:bdfe592802ef939a0e33106ea4a318eeb17822c7ee168c290273cbd5fabd746c \
--hash=sha256:c05557a78f8fa514da0f869556eeda40887a8abc77c76ee3f74cf241778afd5a \
--hash=sha256:c22b1014017111c401469e3acc5433e6acf6ebcc6aa9efb538a533c800971c79 \
--hash=sha256:c9b9e288b4da2f64fd6180644221749de651703e8d0c16bd4b719533a3a7d6e3 \
--hash=sha256:d241cdc4a67b5431c6d7f115fdf63335222414995e3a1df1a41e1182acd4bcc7 \
--hash=sha256:e54c75fd6041f1122476776880f7c3c3295ffa31962dc6ebe2543c00dca58b5d \
--hash=sha256:e8514f4924375f77084e81467e63238b095abda5107620f49421c368a6017ed2 \
--hash=sha256:ee91628c08e76f77b533d65feb3fbe6d9dad699f95be51cf0d022db94089cdc4 \
--hash=sha256:ef46db46c9447103b8f3ff91e8ba009d5fe181b1920a83757a5762551e32bb68 \
--hash=sha256:fa1d16210b6b10a826d71bed68dd9ec24a9e218d5a5e2797f37c573e7ec215ca
fonttools==4.63.0 \
--hash=sha256:032038247a96c1690f9f31e377c389383c902531b085aa4e4dabd6f57f870e69 \
--hash=sha256:063e08bd17bd5a90127a14123de0d6a952dbc847695fd98b63c043d58057f90c \
--hash=sha256:0c18358a155d75034911c5ee397a5b44cd19dd325dbb8b35fb60bf421d6a72ac \
--hash=sha256:0eac00b9118c3c2f87d272e45341871c5b3066baa3c86897fa634a7c3fb59096 \
--hash=sha256:1e874792a8212b44583ea02189d9e693906b2f78b261f372f95d6c563210ac1d \
--hash=sha256:22135da48a348785c5e2d5d2d9d6bec5ed44adacbaeb9db12d9493bf6c6bfa68 \
--hash=sha256:22693918177bd9ceabec4736d338045f357769416fc6b0b2508eefef75b08616 \
--hash=sha256:27fdc65af8da6f88b9c6121c47a464cbe359fcfff7ff6fc2d37a1f395d755b78 \
--hash=sha256:2b8ae05d9eacf6081414d759c0a352769ac28ce31280d6bb8e77b03f9e3c449f \
--hash=sha256:2c14b4fd138c4bafcca294765c547914e1aa431ae1ca94ab99d8db08c958bd3b \
--hash=sha256:308f957cdeaf8abe4e5f2f124902ef405448af92c90f80e302a3b771c2e6116b \
--hash=sha256:37dd23e621e3b0aef1baa70a303b80aaf38449632cfc8fd2a55fb285bbccfc02 \
--hash=sha256:445af2eab030a16b9171ea8bdda7ebf7d96bda2df88ee182a464252f6e05e20d \
--hash=sha256:51394295f1a51de8b5f30bdb1e1b9a4231536c7064ef5c6e211eec19fa36036f \
--hash=sha256:58dc6bb86a78d782f00f9190ca02c119cf5bbe2807536e361e18d42019f877d8 \
--hash=sha256:59ac449f8cca9b4ffa08d2e7bbadad87ce710d69d1eda5c3c1ce579baa987272 \
--hash=sha256:6b2248c5decb223562f7902ff6325077a073f608ee8e33e88ad88db734eb9f49 \
--hash=sha256:6d4741eb179121cab9eea4cb2393d24492373a260d7945006358c08cfbf45419 \
--hash=sha256:6db5140a60a5d731d21ec076745b40a310607731b0a565b50776393188649001 \
--hash=sha256:6e528da43bc3791085f8cb6141b1d13e459226790240340fcbb4625649238b03 \
--hash=sha256:796f27556dbe094c4824f75ca85267e4df776c79036c8441469a4df37038c196 \
--hash=sha256:79cdc9f567aec74a72918fd060283911406750cbc9fd28c1316023deb6ce31a9 \
--hash=sha256:7d76edbff9014094dbf03bd2d074709dfa6ec7aba13d838c937a2b33d2d6a86e \
--hash=sha256:7d782fac32985914c351556f68ac0855391572bcd87de50e05970d3cd4c96fc5 \
--hash=sha256:7dd683fef0663e9f0f45cf541d788d24caa3ec9db50796b588e1757d8b3bc007 \
--hash=sha256:85be818f5506e8a7753153def2c9550178f0ecae6a47b5e0e8dbb23f7cc90380 \
--hash=sha256:948428a275741f0b64b113c955425a953314f4b9ab9997f73a72c83e68e569c8 \
--hash=sha256:9ced0bd02ac751dd6319b0da88aaef24414e3b0dbc32bb4f24944821a3741a27 \
--hash=sha256:9e12f105d2b6342c559c298afb674006bb2893afc7102dcf8a1b55b0486b4e40 \
--hash=sha256:a8b33a82979e0a6a34ff435cc81317be1f95ec1ebb7a3a2d1c8a6a54f02ae44e \
--hash=sha256:a9faff9e0c1f76f9fd55899d2ce785832efebab37eb8ae13995853aef178bef0 \
--hash=sha256:af2fd1664d00a397d75f806985ddb36282091c2131a73a6485c23b4a34722263 \
--hash=sha256:afefc1ed0a59785a7fb06ea7e1678e849c193e1e387db783579bc7b3056fcfcb \
--hash=sha256:b1cd75a03ad8cb5bc40c90bfde68c0c47de423aa19e5c0f362b43520645eea94 \
--hash=sha256:ba04cb5891d4c0c21b6da95eda8d7b090021508a294fff33464fc7d241e0856b \
--hash=sha256:bf00f21eb5fb721dbaf73d1e9da6d02a1af7768f2ebcf9798be98beab8ba90f6 \
--hash=sha256:c0425b277a59cff3d80ca42162a8de360f318438a2ac83570842a678d826d579 \
--hash=sha256:c1aaa4b9c75798400ac043ce04d74e7830376c85095a5a6ed7cba2f17a266bf4 \
--hash=sha256:c2a2a42198b696a6f48fad91709afb55176e66a5e566131219dba372fb7f8c59 \
--hash=sha256:caeb583deeb5168e694b65cda8b4ee62abedfa66cf88488734466f2366b9c4e0 \
--hash=sha256:cb014d58140a38135f16064c74c652ed57aa0b75cbf8bb59cac821f7edb5334e \
--hash=sha256:ccf41f2efdf56994d22d73bef4ced1052161958169428d06ba9724ea9e9a64be \
--hash=sha256:cd7e9857e5e63738b9d9fd707bc1f59c8b09e5177726d23664db393c59bb08bd \
--hash=sha256:d76ac49f929aecaf82d83250b8347e099d7aecba0f4726c1d9b6df3b8bb5fe18 \
--hash=sha256:d7e5c9973aa04c95650c96e5f5ad865fbf42d62079163ecfab1e01cbc2504c22 \
--hash=sha256:dcf076a4474fe0d7367e5bbf5b052c7284fa1feca729c04176ce513521afd8a0 \
--hash=sha256:e3297a6a4059b4acc3a1e9a8b04741f240a80044eef08ebd32e8b5bcdddce75b \
--hash=sha256:ee08ebfa58f6e1aeff5697ab9582105bb620008c1caafb681e4c557e7483027b \
--hash=sha256:ef3048ef05dbb552b89817713d9cac912e00d0fde4a3105c00d29e52e10c89af \
--hash=sha256:fd1e3094f42d806d3d7c79162fc59e5910fcbe3a7360c385b8da969bc4493745
# via weasyprint
identify==2.6.19 \
--hash=sha256:20e6a87f786f768c092a721ad107fc9df0eb89347be9396cadf3f4abbd1fb78a \
@@ -175,193 +190,190 @@ nodeenv==1.10.0 \
--hash=sha256:5bb13e3eed2923615535339b3c620e76779af4cb4c6a90deccc9e36b274d3827 \
--hash=sha256:996c191ad80897d076bdfba80a41994c2b47c68e224c542b48feba42ba00f8bb
# via pre-commit
numpy==2.4.4 \
--hash=sha256:07077278157d02f65c43b1b26a3886bce886f95d20aabd11f87932750dfb14ed \
--hash=sha256:08f2e31ed5e6f04b118e49821397f12767934cfdd12a1ce86a058f91e004ee50 \
--hash=sha256:0aec54fd785890ecca25a6003fd9a5aed47ad607bbac5cd64f836ad8666f4959 \
--hash=sha256:0d35aea54ad1d420c812bfa0385c71cd7cc5bcf7c65fed95fc2cd02fe8c79827 \
--hash=sha256:0d4e437e295f18ec29bc79daf55e8a47a9113df44d66f702f02a293d93a2d6dd \
--hash=sha256:0dfd3f9d3adbe2920b68b5cd3d51444e13a10792ec7154cd0a2f6e74d4ab3233 \
--hash=sha256:1378871da56ca8943c2ba674530924bb8ca40cd228358a3b5f302ad60cf875fc \
--hash=sha256:15716cfef24d3a9762e3acdf87e27f58dc823d1348f765bbea6bef8c639bfa1b \
--hash=sha256:19710a9ca9992d7174e9c52f643d4272dcd1558c5f7af7f6f8190f633bd651a7 \
--hash=sha256:23cbfd4c17357c81021f21540da84ee282b9c8fba38a03b7b9d09ba6b951421e \
--hash=sha256:2483e4584a1cb3092da4470b38866634bafb223cbcd551ee047633fd2584599a \
--hash=sha256:27a8d92cd10f1382a67d7cf4db7ce18341b66438bdd9f691d7b0e48d104c2a9d \
--hash=sha256:28a650663f7314afc3e6ec620f44f333c386aad9f6fc472030865dc0ebb26ee3 \
--hash=sha256:2aa0613a5177c264ff5921051a5719d20095ea586ca88cc802c5c218d1c67d3e \
--hash=sha256:2c194dd721e54ecad9ad387c1d35e63dce5c4450c6dc7dd5611283dda239aabb \
--hash=sha256:2d19e6e2095506d1736b7d80595e0f252d76b89f5e715c35e06e937679ea7d7a \
--hash=sha256:2d390634c5182175533585cc89f3608a4682ccb173cc9bb940b2881c8d6f8fa0 \
--hash=sha256:30caa73029a225b2d40d9fae193e008e24b2026b7ee1a867b7ee8d96ca1a448e \
--hash=sha256:42c16925aa5a02362f986765f9ebabf20de75cdefdca827d14315c568dcab113 \
--hash=sha256:45dbed2ab436a9e826e302fcdcbe9133f9b0006e5af7168afb8963a6520da103 \
--hash=sha256:4636de7fd195197b7535f231b5de9e4b36d2c440b6e566d2e4e4746e6af0ca93 \
--hash=sha256:4a19d9dba1a76618dd86b164d608566f393f8ec6ac7c44f0cc879011c45e65af \
--hash=sha256:4bbc7f303d125971f60ec0aaad5e12c62d0d2c925f0ab1273debd0e4ba37aba5 \
--hash=sha256:4d6d57903571f86180eb98f8f0c839fa9ebbfb031356d87f1361be91e433f5b7 \
--hash=sha256:4e874c976154687c1f71715b034739b45c7711bec81db01914770373d125e392 \
--hash=sha256:51fc224f7ca4d92656d5a5eb315f12eb5fe2c97a66249aa7b5f562528a3be38c \
--hash=sha256:58c8b5929fcb8287cbd6f0a3fae19c6e03a5c48402ae792962ac465224a629a4 \
--hash=sha256:5a285b3b96f951841799528cd1f4f01cd70e7e0204b4abebac9463eecfcf2a40 \
--hash=sha256:5c70f1cc1c4efbe316a572e2d8b9b9cc44e89b95f79ca3331553fbb63716e2bf \
--hash=sha256:62d6b0f03b694173f9fcb1fb317f7222fd0b0b103e784c6549f5e53a27718c44 \
--hash=sha256:6a246d5914aa1c820c9443ddcee9c02bec3e203b0c080349533fae17727dfd1b \
--hash=sha256:6aa3236c78803afbcb255045fbef97a9e25a1f6c9888357d205ddc42f4d6eba5 \
--hash=sha256:6bbe4eb67390b0a0265a2c25458f6b90a409d5d069f1041e6aff1e27e3d9a79e \
--hash=sha256:715d1c092715954784bc79e1174fc2a90093dc4dc84ea15eb14dad8abdcdeb74 \
--hash=sha256:72944b19f2324114e9dc86a159787333b77874143efcf89a5167ef83cfee8af0 \
--hash=sha256:81f4a14bee47aec54f883e0cad2d73986640c1590eb9bfaaba7ad17394481e6e \
--hash=sha256:846300f379b5b12cc769334464656bc882e0735d27d9726568bc932fdc49d5ec \
--hash=sha256:86b6f55f5a352b48d7fbfd2dbc3d5b780b2d79f4d3c121f33eb6efb22e9a2015 \
--hash=sha256:874f200b2a981c647340f841730fc3a2b54c9d940566a3c4149099591e2c4c3d \
--hash=sha256:8a87ec22c87be071b6bdbd27920b129b94f2fc964358ce38f3822635a3e2e03d \
--hash=sha256:8b3b60bb7cba2c8c81837661c488637eee696f59a877788a396d33150c35d842 \
--hash=sha256:8e3ed142f2728df44263aaf5fb1f5b0b99f4070c553a0d7f033be65338329150 \
--hash=sha256:93e15038125dc1e5345d9b5b68aa7f996ec33b98118d18c6ca0d0b7d6198b7e8 \
--hash=sha256:989824e9faf85f96ec9c7761cd8d29c531ad857bfa1daa930cba85baaecf1a9a \
--hash=sha256:99d838547ace2c4aace6c4f76e879ddfe02bb58a80c1549928477862b7a6d6ed \
--hash=sha256:9b2aec6af35c113b05695ebb5749a787acd63cafc83086a05771d1e1cd1e555f \
--hash=sha256:9c585a1790d5436a5374bac930dad6ed244c046ed91b2b2a3634eb2971d21008 \
--hash=sha256:a7164afb23be6e37ad90b2f10426149fd75aee07ca55653d2aa41e66c4ef697e \
--hash=sha256:ac6b31e35612a26483e20750126d30d0941f949426974cace8e6b5c58a3657b0 \
--hash=sha256:ad2e2ef14e0b04e544ea2fa0a36463f847f113d314aa02e5b402fdf910ef309e \
--hash=sha256:b268594bccac7d7cf5844c7732e3f20c50921d94e36d7ec9b79e9857694b1b2f \
--hash=sha256:b5f0362dc928a6ecd9db58868fca5e48485205e3855957bdedea308f8672ea4a \
--hash=sha256:ba1f4fc670ed79f876f70082eff4f9583c15fb9a4b89d6188412de4d18ae2f40 \
--hash=sha256:ba203255017337d39f89bdd58417f03c4426f12beed0440cfd933cb15f8669c7 \
--hash=sha256:c901b15172510173f5cb310eae652908340f8dede90fff9e3bf6c0d8dfd92f83 \
--hash=sha256:c9b39d38a9bd2ae1becd7eac1303d031c5c110ad31f2b319c6e7d98b135c934d \
--hash=sha256:d2a8490669bfe99a233298348acc2d824d496dee0e66e31b66a6022c2ad74a5c \
--hash=sha256:dddbbd259598d7240b18c9d87c56a9d2fb3b02fe266f49a7c101532e78c1d871 \
--hash=sha256:df3775294accfdd75f32c74ae39fcba920c9a378a2fc18a12b6820aa8c1fb502 \
--hash=sha256:e44319a2953c738205bf3354537979eaa3998ed673395b964c1176083dd46252 \
--hash=sha256:e4a010c27ff6f210ff4c6ef34394cd61470d01014439b192ec22552ee867f2a8 \
--hash=sha256:e823b8b6edc81e747526f70f71a9c0a07ac4e7ad13020aa736bb7c9d67196115 \
--hash=sha256:e892aff75639bbef0d2a2cfd55535510df26ff92f63c92cd84ef8d4ba5a5557f \
--hash=sha256:eea7ac5d2dce4189771cedb559c738a71512768210dc4e4753b107a2048b3d0e \
--hash=sha256:ef4059d6e5152fa1a39f888e344c73fdc926e1b2dd58c771d67b0acfbf2aa67d \
--hash=sha256:f169b9a863d34f5d11b8698ead99febeaa17a13ca044961aa8e2662a6c7766a0 \
--hash=sha256:f2cf083b324a467e1ab358c105f6cad5ea950f50524668a80c486ff1db24e119 \
--hash=sha256:f8474c4241bc18b750be2abea9d7a9ec84f46ef861dbacf86a4f6e043401f79e \
--hash=sha256:f983334aea213c99992053ede6168500e5f086ce74fbc4acc3f2b00f5762e9db \
--hash=sha256:f9e75681b59ddaa5e659898085ae0eaea229d054f2ac0c7e563a62205a700121 \
--hash=sha256:fbc356aae7adf9e6336d336b9c8111d390a05df88f1805573ebb0807bd06fd1d \
--hash=sha256:fcfe2045fd2e8f3cb0ce9d4ba6dba6333b8fa05bb8a4939c908cd43322d14c7e
numpy==2.4.6 \
--hash=sha256:001fbb8e08d942dd57599e781f2472269ee7f2755fae407b4f67b2f0b17da3f1 \
--hash=sha256:0280e0356c0829a18d9de1cb7eee50ec22ca639878d7240307ca0943d73cd2c4 \
--hash=sha256:043191bfa8eab18c776647b62723ac9dddece59743b13f49b2016094129c2b3f \
--hash=sha256:06ca2f61ec4385a07a6977c55ba998a4466c123642b4a32694d3128fce18c079 \
--hash=sha256:0a041d3d761dc3c35cc56ce0351506a02bcbc25f7b169f652435141a17db9096 \
--hash=sha256:0ab0a9c4ffb1a6d95ef519fe4247dba8eb6b18ad93999f76b7f657039acabd47 \
--hash=sha256:0c9136e14ed34a9e343a31c533d78a9813a69a3148332bce5e9821cb2f996e66 \
--hash=sha256:110f8b71aacb688ec69062bb7f6938a0f8acb01b7c1c4beb453c65b6d234584d \
--hash=sha256:112b06a867b235ef466ed3508ddf0238050df9c727cafb5301ac385b899189a1 \
--hash=sha256:17f9ade344e7d9b464a084d69bcf18fc691cb1db67c62ed80820bf4926d78f0e \
--hash=sha256:1e254a00cdf42b1e4d5b3d68d33af63268d41340d8885df2ab6470f2e1500147 \
--hash=sha256:1e978ec1e8bd0e0e4de6bb75de9d30cbb74db6b6a2bb727618613703ca0167dd \
--hash=sha256:25c692919ac5a01f170a3bfcd62d745b24fd095c353d50812637d6fcab442e75 \
--hash=sha256:260a5d70215b61ab4fadf5c7baacd64821842975eea312125ed3c39a6391b063 \
--hash=sha256:2803abfebfc990042cd494d8ce2d5f82e9d847af6d35ec486923aa19dbad5e73 \
--hash=sha256:29a287e0cf63ff528da061de6b9f64a4618da591ca1046aafc54062e40ca7eab \
--hash=sha256:29cb7f67d10b479ff07c17d33e39f78c07f71c40ef30d63c153d340e96cd3fb4 \
--hash=sha256:3213d622a0283a39a93d188f3cf72b26862df52fbb4ca3697f51705016523d41 \
--hash=sha256:33111801a01c12a8a1e3721f0a9232f8cfc8ae2c6b7098167e6f623c6073f402 \
--hash=sha256:357cc07a6d7b0b182ff02249616a03742827ebb1277546b5c7cd7f7620a45698 \
--hash=sha256:38efbc8de75c7a0fc1ac190162d892787f3f47b57cc291231aafee36b80982b7 \
--hash=sha256:4081eb135ac24158bd51cdfbef16f1c64df7063b1143f24731387137c092bec8 \
--hash=sha256:40fdc1ae7125e518ea98e53e69a4ebc27e1fd50510c47b7ea130cf21e5e1d42b \
--hash=sha256:4cfe66903cc32a9921a6733d96b19bb6abf310397581bbad89c228f5abaf0ee8 \
--hash=sha256:511dbaf848decaaaf4b4ca48032619fb3138710c4bf7da7617765edad1ef96b0 \
--hash=sha256:55cced7c52e981362f708ad635198e97a752dfba412cc03c23bbf3bd8d5cd662 \
--hash=sha256:56b39e5e0622a09a25bf5baf62f4bcf0cb8a41ae6e2819cf49bbc5a74c083f91 \
--hash=sha256:5dbbdb29840ca3d91ee0fece42fc29278886d908280bfec0a5846c6f901a3eb0 \
--hash=sha256:5f9fb9157b4ce2971008323afe46053787b526ef624fea915b261468a8421a0f \
--hash=sha256:6180d8b35af935aed8ece3a85e0a43f87393ae0ac87c8d2c8bd2c993f7270ef3 \
--hash=sha256:68a5124b13fa6cc2086764a20005d30bc0548146f7f5322f02fce212ca14317f \
--hash=sha256:68bb27509ac1b9a3443094260f6326150663b06abe40b73a2f81160623da5b67 \
--hash=sha256:6f41ae150c4e32db4f3310cdaf64b1593a03dbabe29eec77fc9b50fe64061df6 \
--hash=sha256:7265a2f3d436e54ef9f2b52b5c937e6be778781bd97a590319d7348f1c1ca997 \
--hash=sha256:72fbe16c6fac95aedf5937fa873445cec2110be35d8a4e9433d7501fd98dae6b \
--hash=sha256:7d92c3819208a60205a12a245c91ad70cb0a85336659b19b834205573ac8456e \
--hash=sha256:8155154c7c691289fe18f510b5d4657c68c67989f293f0535a91360392ff6538 \
--hash=sha256:81a1cca95ed5bb92aa8b10dd2cdc9a0d3853a50fad926c28b5d7e8ea54389627 \
--hash=sha256:89cd468399cfd2504718f0ba50e410dca55a170b61a02ad92bb18c8a65186e93 \
--hash=sha256:8ad03c0965fb3c692200e74d458ca28c1dbb4ce96f9a479a8aa041ad5fabca02 \
--hash=sha256:90f9849678c75fe7afa2d348ac842c168b0a4d3d61919687216dfc547976d853 \
--hash=sha256:948424b06129ce883307e8cff868c31396d8dc7630a59c61d70d98dbe70f222c \
--hash=sha256:9cd5ffd25db4e7ba6a375693b3fc0fc1791ec636c17db3720da19bde7180ec43 \
--hash=sha256:a0df0043bdb289bde1f62da130d20df23d58b45429f752bc7a8fc5325a225ecd \
--hash=sha256:a2c306dea656c12c68f51f4cea133cbe78ca7435eb28c735eac1d3ebe73be6e8 \
--hash=sha256:a7830bab239b79cda9c08c2da014761cafb48da6150e1da17ac06283f43b6089 \
--hash=sha256:a7c711e21628b52034bb5ab8d1bce291f752fcc5e92accc615778acee1ff4778 \
--hash=sha256:aaf159caa35993cb1f56fb9b8e4610d35758e7ca005412eb1daa856a78c9c4b1 \
--hash=sha256:ae506e6902902557576a26ff33eda8695e7ecb3cb36c3b573a0765dee114ebdb \
--hash=sha256:b507f5c4c1d508876d1819b6bf9a49d365b96320b5d4993426b33a23ca4b8261 \
--hash=sha256:bf162abab1c1a736333192707cef898e735a5ca00f38f27eeedf44b39d9e85eb \
--hash=sha256:c1a2af6c6ef86344a6b0db6b97834208bf598db514f2b155042439b62605601a \
--hash=sha256:c2d37ab77531417474168eb79d6d80b14f821a966818505d03013d0833edb7a8 \
--hash=sha256:c4fc99836233ea196540b17ab0983aff60ed07941751930f5f4d05bc3b3b7359 \
--hash=sha256:d581b735e177fdcdce6fed8e7e8880a3fb6ee4e3653a3ac6af01c6f4c03effc5 \
--hash=sha256:d6da64deb6b8ed903e7560180a92f2d804ee1ba5eeb849ac2748b8c1aba1f6d7 \
--hash=sha256:d8e8286dd7cea7895157318d1b91cdacac64c479f3cbc8dce548331728484751 \
--hash=sha256:ddea102b48f9e339f3948bf22040944184627a30fdf7f858667673b9c5f033c8 \
--hash=sha256:dfa20cc6ca228e6b155b11da03825975ce66aea520985dbbddf0f2a5a495c605 \
--hash=sha256:e3e5193ef5a3dc73bceee50f7fdc2c90dbb76c42df8d8fae3d1067a583df579e \
--hash=sha256:e3eeb0aabd6bd5ce64faae67e9935203a6991b4bc2a485a767fbafb2c5125f45 \
--hash=sha256:e5805d5a22fd19c8ccff10a9561f9df94436b0545619ea579db2d3c35294bce2 \
--hash=sha256:e85b752a1e912b70eaad4fafbd4d1238007ab221de2009b9a2f5ae7461239895 \
--hash=sha256:eaf7fa2de5c0be8ae6ff8e9bea2ccd725e980541244521d8d4b5f3354a27babe \
--hash=sha256:ebfb099f8dcf083deef3ac1ca4c1503f387cf76296fcb3816b66f5ecb5f54fdb \
--hash=sha256:ece3d2cfe132e7d51f44a832b303895e6f2d499c5e74dfbdb06ee246147a304a \
--hash=sha256:ed9749eef4cbd126da3dc1d6bcb3a57f5eb7ac6a6484146bdbf743f552dfc577 \
--hash=sha256:ede83e07a75dd06bc501566c1eca2afc0d61677c1472ac9ad93fdee6e638a48d \
--hash=sha256:ef4aea96ce4d3b074422cb4f2f64e216bf9e213004bb58ecfdf50ea02ea8eb9a \
--hash=sha256:f3a3570c4a2a16746ac2c31a7c7c7b0c186b95ce902e33db6f28094ed7387dda \
--hash=sha256:f407cb6b8e9d6d8c626bc73c945db1706035af8fd632295547bf1c9e46d092d6 \
--hash=sha256:f74a575920ab21fe304421a3fc28793d82e299cae9eccb37084e9fc7f3617c20
# via opencv-python-headless
opencv-python-headless==4.13.0.92 \
--hash=sha256:0525a3d2c0b46c611e2130b5fdebc94cf404845d8fa64d2f3a3b679572a5bd22 \
--hash=sha256:0bd48544f77c68b2941392fcdf9bcd2b9cdf00e98cb8c29b2455d194763cf99e \
--hash=sha256:1a7d040ac656c11b8c38677cc8cccdc149f98535089dbe5b081e80a4e5903209 \
--hash=sha256:3e0a6f0a37994ec6ce5f59e936be21d5d6384a4556f2d2da9c2f9c5dc948394c \
--hash=sha256:5c8cfc8e87ed452b5cecb9419473ee5560a989859fe1d10d1ce11ae87b09a2cb \
--hash=sha256:77a82fe35ddcec0f62c15f2ba8a12ecc2ed4207c17b0902c7a3151ae29f37fb6 \
--hash=sha256:a7cf08e5b191f4ebb530791acc0825a7986e0d0dee2a3c491184bd8599848a4b \
--hash=sha256:eb60e36b237b1ebd40a912da5384b348df8ed534f6f644d8e0b4f103e272ba7d
opencv-python-headless==5.0.0.93 \
--hash=sha256:030ca5e0837a2963ab36ef896baa9767eb8d2b83353fb28af5a521e40dd8756f \
--hash=sha256:09a872a157c1376ab922a69bbf22f9a95bcc7b658a9d8b436a60212b02b2eeb4 \
--hash=sha256:10818d91510e05c04568ae12b5cd120779c70c01bf897b001a6221fe430df80f \
--hash=sha256:1e55af3abfb462eeeabe5c775f12bdb36216d8a93a3583d69e6bd6e1d6ba7d00 \
--hash=sha256:829717b6a95554f273e49e357cee3b3a2a26b6f4842fbc1bed2b45bdd8f87e0e \
--hash=sha256:840bd717c21e5c11cadadc022a823315ea417f961213d06b4df010e019eb16f4 \
--hash=sha256:b82f9831daab90b725c7c1ee1b36cb5732c367096ac76d119e64e14eb70d5f3c \
--hash=sha256:c6bcd96b185975ea240d22cfdb15a1f6d080cc95264cfbe2621f21bb144d89b9 \
--hash=sha256:ed709fdf9aa0bd1f2ed8549e71d19449b03a675bb581eb292285f6861953be37
# via -r .github/scripts/requirements_dev.in
pdf2image==1.17.0 \
--hash=sha256:eaa959bc116b420dd7ec415fcae49b98100dda3dd18cd2fdfa86d09f112f6d57 \
--hash=sha256:ecdd58d7afb810dffe21ef2b1bbc057ef434dabbac6c33778a38a3f7744a27e2
# via -r .github/scripts/requirements_dev.in
pillow==12.2.0 \
--hash=sha256:00a2865911330191c0b818c59103b58a5e697cae67042366970a6b6f1b20b7f9 \
--hash=sha256:01afa7cf67f74f09523699b4e88c73fb55c13346d212a59a2db1f86b0a63e8c5 \
--hash=sha256:03e7e372d5240cc23e9f07deca4d775c0817bffc641b01e9c3af208dbd300987 \
--hash=sha256:03f6fab9219220f041c74aeaa2939ff0062bd5c364ba9ce037197f4c6d498cd9 \
--hash=sha256:042db20a421b9bafecc4b84a8b6e444686bd9d836c7fd24542db3e7df7baad9b \
--hash=sha256:0538bd5e05efec03ae613fd89c4ce0368ecd2ba239cc25b9f9be7ed426b0af1f \
--hash=sha256:0a34329707af4f73cf1782a36cd2289c0368880654a2c11f027bcee9052d35dd \
--hash=sha256:0c838a5125cee37e68edec915651521191cef1e6aa336b855f495766e77a366e \
--hash=sha256:144748b3af2d1b358d41286056d0003f47cb339b8c43a9ea42f5fea4d8c66b6e \
--hash=sha256:1610dd6c61621ae1cf811bef44d77e149ce3f7b95afe66a4512f8c59f25d9ebe \
--hash=sha256:1e1757442ed87f4912397c6d35a0db6a7b52592156014706f17658ff58bbf795 \
--hash=sha256:22db17c68434de69d8ecfc2fe821569195c0c373b25cccb9cbdacf2c6e53c601 \
--hash=sha256:25373b66e0dd5905ed63fa3cae13c82fbddf3079f2c8bf15c6fb6a35586324c1 \
--hash=sha256:2bb4a8d594eacdfc59d9e5ad972aa8afdd48d584ffd5f13a937a664c3e7db0ed \
--hash=sha256:2c727a6d53cb0018aadd8018c2b938376af27914a68a492f59dfcaca650d5eea \
--hash=sha256:2d192a155bbcec180f8564f693e6fd9bccff5a7af9b32e2e4bf8c9c69dbad6b5 \
--hash=sha256:2e589959f10d9824d39b350472b92f0ce3b443c0a3442ebf41c40cb8361c5b97 \
--hash=sha256:2e5a76d03a6c6dcef67edabda7a52494afa4035021a79c8558e14af25313d453 \
--hash=sha256:325ca0528c6788d2a6c3d40e3568639398137346c3d6e66bb61db96b96511c98 \
--hash=sha256:34c0d99ecccea270c04882cb3b86e7b57296079c9a4aff88cb3b33563d95afaa \
--hash=sha256:390ede346628ccc626e5730107cde16c42d3836b89662a115a921f28440e6a3b \
--hash=sha256:394167b21da716608eac917c60aa9b969421b5dcbbe02ae7f013e7b85811c69d \
--hash=sha256:3997232e10d2920a68d25191392e3a4487d8183039e1c74c2297f00ed1c50705 \
--hash=sha256:3adc9215e8be0448ed6e814966ecf3d9952f0ea40eb14e89a102b87f450660d8 \
--hash=sha256:3e080565d8d7c671db5802eedfb438e5565ffa40115216eabb8cd52d0ecce024 \
--hash=sha256:4a6c9fa44005fa37a91ebfc95d081e8079757d2e904b27103f4f5fa6f0bf78c0 \
--hash=sha256:4bfd07bc812fbd20395212969e41931001fd59eb55a60658b0e5710872e95286 \
--hash=sha256:4e6c62e9d237e9b65fac06857d511e90d8461a32adcc1b9065ea0c0fa3a28150 \
--hash=sha256:50d8520da2a6ce0af445fa6d648c4273c3eeefbc32d7ce049f22e8b5c3daecc2 \
--hash=sha256:51c4167c34b0d8ba05b547a3bb23578d0ba17b80a5593f93bd8ecb123dd336a3 \
--hash=sha256:56a3f9c60a13133a98ecff6197af34d7824de9b7b38c3654861a725c970c197b \
--hash=sha256:56b25336f502b6ed02e889f4ece894a72612fe885889a6e8c4c80239ff6e5f5f \
--hash=sha256:57850958fe9c751670e49b2cecf6294acc99e562531f4bd317fa5ddee2068463 \
--hash=sha256:58f62cc0f00fd29e64b29f4fd923ffdb3859c9f9e6105bfc37ba1d08994e8940 \
--hash=sha256:5c0a9f29ca8e79f09de89293f82fc9b0270bb4af1d58bc98f540cc4aedf03166 \
--hash=sha256:5cdfebd752ec52bf5bb4e35d9c64b40826bc5b40a13df7c3cda20a2c03a0f5ed \
--hash=sha256:5d04bfa02cc2d23b497d1e90a0f927070043f6cbf303e738300532379a4b4e0f \
--hash=sha256:5d2fd0fa6b5d9d1de415060363433f28da8b1526c1c129020435e186794b3795 \
--hash=sha256:62f5409336adb0663b7caa0da5c7d9e7bdbaae9ce761d34669420c2a801b2780 \
--hash=sha256:632ff19b2778e43162304d50da0181ce24ac5bb8180122cbe1bf4673428328c7 \
--hash=sha256:6562ace0d3fb5f20ed7290f1f929cae41b25ae29528f2af1722966a0a02e2aa1 \
--hash=sha256:673aa32138f3e7531ccdbca7b3901dba9b70940a19ccecc6a37c77d5fdeb05b5 \
--hash=sha256:6a6e67ea2e6feda684ed370f9a1c52e7a243631c025ba42149a2cc5934dec295 \
--hash=sha256:6a9adfc6d24b10f89588096364cc726174118c62130c817c2837c60cf08a392b \
--hash=sha256:6bb77b2dcb06b20f9f4b4a8454caa581cd4dd0643a08bacf821216a16d9c8354 \
--hash=sha256:6e6b2a0c538fc200b38ff9eb6628228b77908c319a005815f2dde585a0664b60 \
--hash=sha256:71cde9a1e1551df7d34a25462fc60325e8a11a82cc2e2f54578e5e9a1e153d65 \
--hash=sha256:7371b48c4fa448d20d2714c9a1f775a81155050d383333e0a6c15b1123dda005 \
--hash=sha256:766cef22385fa1091258ad7e6216792b156dc16d8d3fa607e7545b2b72061f1c \
--hash=sha256:7b14cc0106cd9aecda615dd6903840a058b4700fcb817687d0ee4fc8b6e389be \
--hash=sha256:7f84204dee22a783350679a0333981df803dac21a0190d706a50475e361c93f5 \
--hash=sha256:8023abc91fba39036dbce14a7d6535632f99c0b857807cbbbf21ecc9f4717f06 \
--hash=sha256:80b2da48193b2f33ed0c32c38140f9d3186583ce7d516526d462645fd98660ae \
--hash=sha256:8297651f5b5679c19968abefd6bb84d95fe30ef712eb1b2d9b2d31ca61267f4c \
--hash=sha256:88d387ff40b3ff7c274947ed3125dedf5262ec6919d83946753b5f3d7c67ea4c \
--hash=sha256:88ddbc66737e277852913bd1e07c150cc7bb124539f94c4e2df5344494e0a612 \
--hash=sha256:8bd7903a5f2a4545f6fd5935c90058b89d30045568985a71c79f5fd6edf9b91e \
--hash=sha256:8be29e59487a79f173507c30ddf57e733a357f67881430449bb32614075a40ab \
--hash=sha256:8c984051042858021a54926eb597d6ee3012393ce9c181814115df4c60b9a808 \
--hash=sha256:8cbeb542b2ebc6fcdacabf8aca8c1a97c9b3ad3927d46b8723f9d4f033288a0f \
--hash=sha256:8e9c4f5b3c546fa3458a29ab22646c1c6c787ea8f5ef51300e5a60300736905e \
--hash=sha256:90e6f81de50ad6b534cab6e5aef77ff6e37722b2f5d908686f4a5c9eba17a909 \
--hash=sha256:975385f4776fafde056abb318f612ef6285b10a1f12b8570f3647ad0d74b48ec \
--hash=sha256:9a8a34cc89c67a65ea7437ce257cea81a9dad65b29805f3ecee8c8fe8ff25ffe \
--hash=sha256:9aba9a17b623ef750a4d11b742cbafffeb48a869821252b30ee21b5e91392c50 \
--hash=sha256:9f08483a632889536b8139663db60f6724bfcb443c96f1b18855860d7d5c0fd4 \
--hash=sha256:a4e8f36e677d3336f35089648c8955c51c6d386a13cf6ee9c189c5f5bd713a9f \
--hash=sha256:a52edc8bfff4429aaabdf4d9ee0daadbbf8562364f940937b941f87a4290f5ff \
--hash=sha256:a830b1a40919539d07806aa58e1b114df53ddd43213d9c8b75847eee6c0182b5 \
--hash=sha256:aa88ccfe4e32d362816319ed727a004423aab09c5cea43c01a4b435643fa34eb \
--hash=sha256:af73337013e0b3b46f175e79492d96845b16126ddf79c438d7ea7ff27783a414 \
--hash=sha256:b1c1fbd8a5a1af3412a0810d060a78b5136ec0836c8a4ef9aa11807f2a22f4e1 \
--hash=sha256:b85f66ae9eb53e860a873b858b789217ba505e5e405a24b85c0464822fe88032 \
--hash=sha256:b86024e52a1b269467a802258c25521e6d742349d760728092e1bc2d135b4d76 \
--hash=sha256:bd9c0c7a0c681a347b3194c500cb1e6ca9cab053ea4d82a5cf45b6b754560136 \
--hash=sha256:bfa9c230d2fe991bed5318a5f119bd6780cda2915cca595393649fc118ab895e \
--hash=sha256:d362d1878f00c142b7e1a16e6e5e780f02be8195123f164edf7eddd911eefe7c \
--hash=sha256:d5d38f1411c0ed9f97bcb49b7bd59b6b7c314e0e27420e34d99d844b9ce3b6f3 \
--hash=sha256:dac8d77255a37e81a2efcbd1fc05f1c15ee82200e6c240d7e127e25e365c39ea \
--hash=sha256:dd025009355c926a84a612fecf58bb315a3f6814b17ead51a8e48d3823d9087f \
--hash=sha256:deede7c263feb25dba4e82ea23058a235dcc2fe1f6021025dc71f2b618e26104 \
--hash=sha256:e74473c875d78b8e9d5da2a70f7099549f9eb37ded4e2f6a463e60125bccd176 \
--hash=sha256:ee3120ae9dff32f121610bb08e4313be87e03efeadfc6c0d18f89127e24d0c24 \
--hash=sha256:eedf4b74eda2b5a4b2b2fb4c006d6295df3bf29e459e198c90ea48e130dc75c3 \
--hash=sha256:efd8c21c98c5cc60653bcb311bef2ce0401642b7ce9d09e03a7da87c878289d4 \
--hash=sha256:f1c943e96e85df3d3478f7b691f229887e143f81fedab9b20205349ab04d73ed \
--hash=sha256:f278f034eb75b4e8a13a54a876cc4a5ab39173d2cdd93a638e1b467fc545ac43 \
--hash=sha256:f3f40b3c5a968281fd507d519e444c35f0ff171237f4fdde090dd60699458421 \
--hash=sha256:f490f9368b6fc026f021db16d7ec2fbf7d89e2edb42e8ec09d2c60505f5729c7 \
--hash=sha256:fb043ee2f06b41473269765c2feae53fc2e2fbf96e5e22ca94fb5ad677856f06 \
--hash=sha256:fc3d34d4a8fbec3e88a79b92e5465e0f9b842b628675850d860b8bd300b159f5
pillow==12.3.0 \
--hash=sha256:00808c5e14ef63ac5161091d242999076604ff74b883423a11e5d7bbb38bf756 \
--hash=sha256:04f01d28a6aaff387bf842a13be313df23ba0597a44f1a976c9feb3c6ff4711a \
--hash=sha256:06ff022112bc9cbf83b60f8e028d94ad87b60621706487e65f673de61610ab59 \
--hash=sha256:0740a512dc522224c77d9aa5a8d70d8b7d73fb91f2c21125d8d025d3b8990e45 \
--hash=sha256:0847a763afefb695bc912d7c131e7e0632d4edc1d8698f58ddabec8e46b8b6d3 \
--hash=sha256:0dd2064cbc55aaec028ef5fbb60fa47bb6c3e7918e07ff17935284b227a9d2df \
--hash=sha256:0feb2e9d6ad6c9e3c06effe9d00f3f1e618a6643273576b016f591e9315a7139 \
--hash=sha256:10e41f0fbf1eec8cfd234b8fe17a4caac7c9d0db4c204d3c173a8f9f6ef3232b \
--hash=sha256:1182d52bc2d5e5d7d0949503aa7e36d12f42205dc287e4883f407b1988820d39 \
--hash=sha256:164b31cd1a0490ab6efae01aa5df49da7061be0af1b30e035b6e9a1bfe34ee6e \
--hash=sha256:1657923d2d45afb66526e5b933e5b3052e6bdea196c90d3abb2424e18c77dae8 \
--hash=sha256:186941b6aef820ad110fb01fb06eb925374dc3a21b17e37ec9a53b250c6fe2d1 \
--hash=sha256:1cca606cd25738df4ed873d5ad46bbdb3d83b5cbca291f6b4ff13a4df6b0bbe8 \
--hash=sha256:21900ce7ba264168cd50defae43cd75d25c833ad4ad6e73ffc5596d12e25ac89 \
--hash=sha256:236ff70b9312fb68943c703aa842ca6a758abfa45ac187a5e7c1452e96ef72b5 \
--hash=sha256:23aceaa007d6172b02c277f0cd359c79492bbb14f7072b4ede9fbcaf20648130 \
--hash=sha256:23d27a3e0307ec2244cc51e7287b919aa68d097504ebe19df4e76a98a3eea5bd \
--hash=sha256:24870b09b224f7ae3c39ed07d10e819d06f8720bc551847b1d623832b5b0e28d \
--hash=sha256:251bf95b67017e27b13d82f5b326234ca62d70f9cf4c2b9032de2358a3b12c7b \
--hash=sha256:25b9b82bb22e6e2b3cd07b39c68b7b862001226cb3dff7130d1cb914121b39ed \
--hash=sha256:28ce87c5ab450a9dd970b52e5aca5fe63ed432d18a2eaddd1979a00a1ba24ace \
--hash=sha256:300557495eb45ebb8aec96c2da9c4be642fbf7cd937278b4013ba894ea8eb0eb \
--hash=sha256:30f2aa603c41533cc25c05acd0da21636e84a315768feb631c937177db558931 \
--hash=sha256:331b624368d4f1d069149002f25f44bc61c8919ce8ddb3c45bdad8f6e2d89510 \
--hash=sha256:37d6d0a00072fd2948eb22bce7e1475f34569d90c87c59f7a2ec59541b77f7a6 \
--hash=sha256:37dc8f7bbb66efe481bb60defacef820c950c24713fb44962ed6aa2a50966de1 \
--hash=sha256:3b8182a766685eaa002637e28b4ec8d6b18819a0c71f579bf0dbaa5830297cce \
--hash=sha256:3edce1d53195db527e0191f84b71d02022de0540bf43a16ed734ed7537b07385 \
--hash=sha256:446c34dcc4324b084a53b705127dc15717b22c5e140ae0a3c38349d4efec071e \
--hash=sha256:4998562bf62a445225f22e07c896bb04b35b1b1f2eb6d760584c9c51d7a5f78c \
--hash=sha256:4b0a7fe987b14c31ebda6083f74f22b561fd3739bc0ac51e019622e3d72668c7 \
--hash=sha256:4e8c2a84d977f50b9daed6eeaf3baef67d00d5d74d932288f02cb94518ee3ace \
--hash=sha256:4f883547d4b7f0495ebe7056b0cc2aea76094e7a4abc8e933540f3271df27d9c \
--hash=sha256:514435a37670e3e5e08f3945b68718b6ed329bb84367777e16f9f4dfe1e61a0f \
--hash=sha256:53aa02d20d10c3d814d536aa4e5ac9b84ca0ff5a88377963b085ad6822f93e64 \
--hash=sha256:5594fc43d548a7ed94949d139aa1341b270f1863f11cfd37f5a6c8b778a6b67f \
--hash=sha256:571b9fcb07b97ef3a492028fb3d2dc0993ca23a06138b0315286566d29ef718a \
--hash=sha256:57b3d78c95ba9059768b10e28b813002261d3f3dfc55cc48b0c988f625175827 \
--hash=sha256:5afb51d599ea772b8365ae807ae557f18bccfe46ab261fd1c2a9ed700fc6eb17 \
--hash=sha256:6b02afb9b97f65fbca5f31db6a2a3ba21aa93030225f150fa3f249717e938fb4 \
--hash=sha256:6c0016e7b354317c4e9e525b937ac8596c38d2d232b419529b9cd7a1cd46e39a \
--hash=sha256:71d6097b330eea8fd15097780c8e89cb1a8ce7838669f48c5bacd6f663dd4701 \
--hash=sha256:756c768d0c9c2955feb7a56c37ea24aea2e369f8d36a88da270b6a9f19e62b5e \
--hash=sha256:78cb2c6865a35ab8ff8b75fd122f6033b92a62c82801110e48ddd6c936a45d91 \
--hash=sha256:7a743ff716f746fc19a9557f60dab1600d4613255f8a7aeb3cdde4db7eb15a66 \
--hash=sha256:85f998ea1848bc6757289e739cfbdda3a04adfd58b02fc018ce54d754a5ce468 \
--hash=sha256:8728f216dcdb6e6d555cf971cb34076139ad74b31fc2c14da4fafc741c5f6217 \
--hash=sha256:877c3f311ff35410f690861c4409e7ccbf0cd2f878e50628a28e5a0bb689e658 \
--hash=sha256:8cd2f7bdda092d99c9fc2fb7391354f306d01443d22785d0cbfafa2e2c8bb418 \
--hash=sha256:8e95e1385e4998ae9694eeaa4730ba5457ff61185b3a55e2e7bea0880aef452a \
--hash=sha256:962864dc93511324d51ddbb5b9f8731bf71675b93ca612a07441896f4688fb8c \
--hash=sha256:9cf95fe4d0f84c82d282745d9bb08ad9f926efa00be4697e767b814ce40d4330 \
--hash=sha256:9e881fca225083806662a5c43d627d215f258ff43c890f831966c7d7ba9c7402 \
--hash=sha256:a2b55dd6b2a4c4b7d87ffa56bdb33fdc5fdb9a462173861a7bc097f17d91cb09 \
--hash=sha256:a45650e8ce7fafffd731db8550230db6b0d306d181a90b67d3e6bca2f1990930 \
--hash=sha256:a876864214e136f0eb367788dbd7df045f4806801518e2cfe9e13229cfe06d8f \
--hash=sha256:ae26d61dfa7a47befdc7572b521024e8745f3d809bd95ca9505a7bba9ef849ec \
--hash=sha256:af8d94b0db561cf68b88a267c5c44b49e134f525d0dc2cb7ed413a66bc23559a \
--hash=sha256:b343699e8308bdc51978310e1c959c584e7869cc8c40780058c87da7781a1e94 \
--hash=sha256:b3c777e849237620b022f7f297dd67705f9f5cf1685f09f02e46f93e92725468 \
--hash=sha256:b629de27fda84b42cde7edef0d85f13b958b47f6e9bbcbba9b673c562a89bd8b \
--hash=sha256:ba09209fbe443b4acccebe845d8a138b89a8f4fbaeedd44953490b5315d5e965 \
--hash=sha256:ba54cfebe86920a559a7c4d6b9050791c20513650a1952ebe3368c7dc70306f8 \
--hash=sha256:bcb46e2f9feff8d06323983bd83ed00c201fdcab3d74973e7072a889b3979fcd \
--hash=sha256:bcc33feacfaefce60c12fd500a277533bdc02b10a19f7f6d348763d8140bbba7 \
--hash=sha256:bf16ba1b4d0b6b7c8e534936632270cf70eb00dbe09005bc345b2677b726855c \
--hash=sha256:cf1845d02ad822a369a49f2bb9345b1614744267682e7a03527dc3bf6eea1777 \
--hash=sha256:d69141514cc30b774ceea5e3ed3a6635c8d8a96edf664689b890f4089111fb35 \
--hash=sha256:d9c7f76c0673154f044e9d78c8655fb4213f6ca31a836df48b40fe5d187717b9 \
--hash=sha256:dbce0b29841537a2fa4a214c2bbf14de3587c9680caa9b4e217568472490b28f \
--hash=sha256:dc624f6bc473dacdf7ef7eb8678d0d08edf15cd94fad6ae5c7d6cc67a4e4902f \
--hash=sha256:e158cb00350dc278f3b91551101aa7d12415a66ebf2c91d8d5ac14e56ddd3ad0 \
--hash=sha256:e491916b378fba47242221bb9ead245211b70d504f495d105d17b14a24b4907c \
--hash=sha256:e795b7eb908249c4e43c7c99fac7c2c75dab0c43566e37db472a355f63693d71 \
--hash=sha256:e7e480451b9fa137494bccd3a7d69adbe8ac65a87d97be61e11f1b1050a5bac3 \
--hash=sha256:e91206ee562682b51b98ef4b26a6ef48fd84e15fd4c4bc5ec768eb641d206838 \
--hash=sha256:e9871b1ffbfa9656b60aeee92ed5136a5742696006fa322b29ea3d8da0ecc9cf \
--hash=sha256:e9aeb04d6aef139de265b29683e119b638208f88cf73cdd1658aa07221165321 \
--hash=sha256:ebaea975e03d3141d9d3a507df75c9b3ec90fa9d2ffd07567b3a978d9d790b26 \
--hash=sha256:f0606c8bf2cdefea14a43530f7657cbbb7ecf1c4222512492ef4a4434a9501ec \
--hash=sha256:f13c32a3abd6079a66d9526e18dad9b6d280384d49d7c54040cd57b6424041d9 \
--hash=sha256:f7401aebd7f581d7f83a439d87d474999317ee099218e5ad25d125290990ba65 \
--hash=sha256:fa4ecea169a355be7a3ade2c783e2ed12f0e40d2c5621cda8b3297faf7fbb9f5 \
--hash=sha256:fbd139c8447d25dd750ab79ee274cc5e1fe80fc56340ab10b18a195e1b6eca3e \
--hash=sha256:fdafc9cce40277e0f7a0feabce0ee50dd2fa1800f3b38015e51296b5e814048d \
--hash=sha256:fe3cca2e4e8a592be0f269a1ca4835c25199d9f3ce815c8491048f785b0a0198 \
--hash=sha256:ffd0c5368496f41b0944be820fcb7a838aa6e623d250b01acf2643939c3f99d7
# via
# -r .github/scripts/requirements_dev.in
# pdf2image
# weasyprint
platformdirs==4.9.6 \
--hash=sha256:3bfa75b0ad0db84096ae777218481852c0ebc6c727b3168c1b9e0118e458cf0a \
--hash=sha256:e61adb1d5e5cb3441b4b7710bea7e4c12250ca49439228cc1021c00dcfac0917
platformdirs==4.10.0 \
--hash=sha256:31e761a6a0ca04faf7353ea759bdba55652be214725111e5aac52dfa29d4bef7 \
--hash=sha256:fb516cdb12eb0d857d0cd85a7c57cea4d060bee4578d6cf5a14dfdf8cbf8784a
# via
# python-discovery
# virtualenv
@@ -381,9 +393,9 @@ pyphen==0.17.2 \
--hash=sha256:3a07fb017cb2341e1d9ff31b8634efb1ae4dc4b130468c7c39dd3d32e7c3affd \
--hash=sha256:f60647a9c9b30ec6c59910097af82bc5dd2d36576b918e44148d8b07ef3b4aa3
# via weasyprint
python-discovery==1.2.2 \
--hash=sha256:876e9c57139eb757cb5878cbdd9ae5379e5d96266c99ef731119e04fffe533bb \
--hash=sha256:e1ae95d9af875e78f15e19aed0c6137ab1bb49c200f21f5061786490c9585c7a
python-discovery==1.4.4 \
--hash=sha256:5cad33982d412c1f3ffb8f9ca4ea292c9680bca3942451d30b69c37fce53a4a3 \
--hash=sha256:abebe9120b43453b68c908acfb1e72a19d1a959ed2cb620ad38fc57d08056dbe
# via virtualenv
pyyaml==6.0.3 \
--hash=sha256:00c4bdeba853cc34e7dd471f16b4114f4162dc03e6b7afcc2128711f0eca823c \
@@ -470,17 +482,17 @@ tinyhtml5==2.1.0 \
--hash=sha256:60a50ec3d938a37e491efa01af895853060943dcebb5627de5b10d188b338a67 \
--hash=sha256:6e11cfff38515834268daf89d5f85bbde0b6dd02e8d9e212d1385c2289b89f0a
# via weasyprint
unoserver==3.6 \
--hash=sha256:25c360fa194396a89cb79b4edd2735f8e4f0fd8531e59db3952114585bd7df05 \
--hash=sha256:e446bcb3638c51880f002aaeecab1cf74dfa9df81035f027f7ff2e081b6d7015
unoserver==3.7 \
--hash=sha256:b05f9578506ac7374ae1b314c3a79528636c542ac78220a9ce99110584ca424b \
--hash=sha256:fc44e6808071c9d2957e705ecf1742cea8a582aa5d5cc23babf36bb332ec6e8e
# via -r .github/scripts/requirements_dev.in
virtualenv==21.2.4 \
--hash=sha256:29d21e941795206138d0f22f4e45ff7050e5da6c6472299fb7103318763861ac \
--hash=sha256:b294ef68192638004d72524ce7ef303e9d0cf5a44c95ce2e54a7500a6381cada
virtualenv==21.6.1 \
--hash=sha256:15f978b7cd329f24855ff4a0c4b4899cc7678589f49adbdcbbb4d3232e641128 \
--hash=sha256:afe991df855715a2b2f60edfcc0107ef95a79fdfd8cb4cdaa71603d1c12e463b
# via pre-commit
weasyprint==68.1 \
--hash=sha256:4dc3ba63c68bbbce3e9617cb2226251c372f5ee90a8a484503b1c099da9cf5be \
--hash=sha256:d3b752049b453a5c95edb27ce78d69e9319af5a34f257fa0f4c738c701b4184e
weasyprint==69.0 \
--hash=sha256:475951cfd917014de6d4d005caff48c6aa867e7e42b80cd5b16a0484a1609ee6 \
--hash=sha256:a7a32f39ca16bd82ef11de99c92ea4b5f14951c9033af035e451ce4f4ee0a88c
# via -r .github/scripts/requirements_dev.in
webencodings==0.5.1 \
--hash=sha256:a0af1213f3c2226497a97e2b3aa01a7e4bee4f403f95be16fc9acd2947514a78 \
@@ -489,27 +501,28 @@ webencodings==0.5.1 \
# cssselect2
# tinycss2
# tinyhtml5
zopfli==0.4.1 \
--hash=sha256:02086247dd12fda929f9bfe8b3962b6bcdbfc8c82e99255aebcf367867cf0760 \
--hash=sha256:07a5cdc5d1aaa6c288c5d9f5a5383042ba743641abf8e2fd898dcad622d8a38e \
--hash=sha256:27823dc1161a4031d1c25925fd45d9868ec0cbc7692341830a7dcfa25063662c \
--hash=sha256:2f992ac7d83cbddd889e1813ace576cbc91a05d5d7a0a21b366e2e5f492e7707 \
--hash=sha256:4238d4d746d1095e29c9125490985e0c12ffd3654f54a24af551e2391e936d54 \
--hash=sha256:5a4c22b6161f47f5bd34637dbaee6735abd287cd64e0d1ce28ef1871bf625f4b \
--hash=sha256:84a31ba9edc921b1d3a4449929394a993888f32d70de3a3617800c428a947b9b \
--hash=sha256:a899eca405662a23ae75054affa3517a060362eae1185d3d791c86a50153c4dd \
--hash=sha256:a93c2ecafff372de6c0aa2212eff18a75f6c71a100372fee7b4b129cc0b6f9a7 \
--hash=sha256:cb136a74d14a4ecfae29cb0fdecece58a6c115abc9a74c12bc6ac62e80f229d7 \
--hash=sha256:d7bcee1b189d64ec33d1e05cfa1b6a1268c29329c382f6ca1bd6245b04925c57 \
--hash=sha256:fdfb7ce9f5de37a5b2f75dd2642fd7717956ef2a72e0387302a36d382440db07
zopfli==0.4.3 \
--hash=sha256:0087c9a6f0c8a052be0f6d1a9bb71b6caffdd3e10201d6d6166e28d482cebe6d \
--hash=sha256:47604eee5c6704bdf0e94d8391fe3b74ddb2abd84128fbcfdc3ee0fc265feaef \
--hash=sha256:62248dbf8dbcbd588ee194b210e5be9fa80bce29641f55599d6d394bd2a9d8a3 \
--hash=sha256:628c3e941752880b3491db8d44163d0aedb221944e22a17187ff7fc549b050f6 \
--hash=sha256:769875152d0625c46707bcca57d4b2233fe653482067acd55fbf6ec525cb9bdc \
--hash=sha256:7e9703ca6e7ef66c8d05e0826b6f558b680c9db8206f84f05a3ee93430a12e42 \
--hash=sha256:7fa3c35193475290e3f007bbcdebdbae64ba2f012d75c632da0d727e1da50d5e \
--hash=sha256:88f4fbe429aad72bc206275d81fab11a097e0f951a5848d1f51083c37ea73073 \
--hash=sha256:921c2c9907f4364963848da5ad194b46d68865e07fdb975d04fd09bc42d47357 \
--hash=sha256:d3a50f91a13cea9bafe025de8fd87a005eb26de02a4f0c193127ddbf23ac8ebe \
--hash=sha256:d4f51dd1ab5312e837e2091284e0d9f1a138188f2e65812f9a5799dc02c45f94 \
--hash=sha256:eb0c9c1d40a8cb1d58762d7e57290ccb753e0828c4d01be8acb59aae5d0ca206 \
--hash=sha256:f2e0adcf7d36c6fd0dd36cc771ef7f0c5803a05666feafcd90d7170174a4148e
# via fonttools
# The following packages are considered to be unsafe in a requirements file:
pip==26.0.1 \
--hash=sha256:bdb1b08f4274833d62c1aa29e20907365a2ceb950410df15fc9521bad440122b \
--hash=sha256:c4037d8a277c89b320abe636d59f91e6d0922d08a05b60e85e53b296613346d8
pip==26.1.2 \
--hash=sha256:382ff9f685ee3bc25864f820aa50505825f10f5458ffff07e30a6d96e5715cab \
--hash=sha256:f49cd134c61cf2fd75e0ce2676db03e4054504a5a4986d00f8299ae632dc4605
# via -r .github/scripts/requirements_dev.in
setuptools==82.0.1 \
--hash=sha256:7d872682c5d01cfde07da7bccc7b65469d3dca203318515ada1de5eda35efbf9 \
--hash=sha256:a59e362652f08dcd477c78bb6e7bd9d80a7995bc73ce773050228a348ce2e5bb
setuptools==83.0.0 \
--hash=sha256:025bccbbf0fa05b6192bc64ae1e7b16e001fd6d6d4d5de03c97b1c1ade523bef \
--hash=sha256:29b23c360f22f414dc7336bb39178cc7bcbf6021ed2733cde173f09dba19abb3
# via -r .github/scripts/requirements_dev.in
.github/scripts/requirements_sync_readme.txt
+4 -4
View File
@@ -1,5 +1,5 @@
#
# This file is autogenerated by pip-compile with Python 3.12
# This file is autogenerated by pip-compile with Python 3.13
# by the following command:
#
# pip-compile --generate-hashes --output-file='.github\scripts\requirements_sync_readme.txt' --strip-extras '.github\scripts\requirements_sync_readme.in'
@@ -8,7 +8,7 @@ tomli-w==1.2.0 \
--hash=sha256:188306098d013b691fcadc011abd66727d3c414c571bb01b1a174ba8c983cf90 \
--hash=sha256:2dd14fac5a47c27be9cd4c976af5a12d87fb1f0b4512f81d69cce3b35ae25021
# via -r .github/scripts/requirements_sync_readme.in
tomlkit==0.14.0 \
--hash=sha256:592064ed85b40fa213469f81ac584f67a4f2992509a7c3ea2d632208623a3680 \
--hash=sha256:cf00efca415dbd57575befb1f6634c4f42d2d87dbba376128adb42c121b87064
tomlkit==0.15.0 \
--hash=sha256:4dbc8f0fc024412b57ced8757ac7461305126a648ff8c2c807fcb8e133a78738 \
--hash=sha256:7d1a9ecba3086638211b13814ea79c90dd54dd11993564376f3aa92271f5c7a3
# via -r .github/scripts/requirements_sync_readme.in
.github/workflows/PR-Auto-Deploy-V2.yml
+6 -34
View File
@@ -23,13 +23,9 @@ permissions:
pull-requests: write
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
check-pr:
if: (github.event_name == 'pull_request' && github.event.action != 'closed') || github.event_name == 'workflow_dispatch'
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
runs-on: ubuntu-latest
outputs:
should_deploy: ${{ steps.decide.outputs.should_deploy }}
is_fork: ${{ steps.resolve.outputs.is_fork }}
@@ -101,8 +97,8 @@ jobs:
echo "allow_fork=${allow_fork:-false}" >> $GITHUB_OUTPUT
deploy-v2-pr:
needs: [pick, check-pr]
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
needs: check-pr
runs-on: ubuntu-latest
if: needs.check-pr.outputs.should_deploy == 'true' && (needs.check-pr.outputs.is_fork == 'false' || needs.check-pr.outputs.allow_fork == 'true')
# Concurrency control - only one deployment per PR at a time
concurrency:
@@ -112,10 +108,7 @@ jobs:
contents: read
issues: write
pull-requests: write
id-token: write
env:
USE_DEPOT: ${{ needs.pick.outputs.is_fork != 'true' }}
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
# Single source of truth for whether this preview embeds the admin portal:
# drives the image build-arg and the deployment comment.
BUILD_PORTAL: "true"
@@ -190,12 +183,7 @@ jobs:
token: ${{ secrets.GITHUB_TOKEN }}
fetch-depth: 0 # Fetch full history for commit hash detection
- name: Set up Depot CLI
if: env.USE_DEPOT == 'true'
uses: depot/setup-action@15c09a5f77a0840ad4bce955686522a257853461 # v1.0.0
- name: Set up Docker Buildx
if: env.USE_DEPOT != 'true'
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
- name: Get version number
@@ -240,22 +228,8 @@ jobs:
echo "Image needs to be built"
fi
- name: Build and push V2 image (Depot)
if: env.USE_DEPOT == 'true' && steps.check-image.outputs.exists == 'false'
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: .
file: ./docker/embedded/Dockerfile
push: true
tags: ${{ secrets.DOCKER_HUB_USERNAME }}/test:v2-${{ steps.commit-hash.outputs.app_short }}
build-args: |
VERSION_TAG=v2-alpha
BUILD_PORTAL=${{ env.BUILD_PORTAL }}
platforms: linux/amd64
- name: Build and push V2 image (Docker fork fallback)
if: env.USE_DEPOT != 'true' && steps.check-image.outputs.exists == 'false'
- name: Build and push V2 image
if: steps.check-image.outputs.exists == 'false'
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
with:
context: .
@@ -297,7 +271,6 @@ jobs:
- /stirling/V2-PR-${{ needs.check-pr.outputs.pr_number }}/storage:/storage:rw
environment:
DISABLE_ADDITIONAL_FEATURES: "false"
POLICIES_ENABLED: "true"
STIRLING_BILLING_ACCOUNT_LINK_ENABLED: "true"
SECURITY_ENABLELOGIN: "true"
SECURITY_INITIALLOGIN_USERNAME: "${{ secrets.TEST_LOGIN_USERNAME }}"
@@ -475,8 +448,7 @@ jobs:
cleanup-v2-deployment:
if: github.event.action == 'closed'
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
runs-on: ubuntu-latest
permissions:
contents: read
issues: write
.github/workflows/PR-Demo-Comment-with-react.yml
+9 -49
View File
@@ -34,12 +34,8 @@ permissions:
pull-requests: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
check-comment:
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
runs-on: ubuntu-latest
permissions:
issues: write
if: |
@@ -179,15 +175,11 @@ jobs:
}
deploy-pr:
needs: [pick, check-comment]
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
needs: check-comment
runs-on: ubuntu-latest
permissions:
issues: write
pull-requests: write
id-token: write
env:
USE_DEPOT: ${{ needs.pick.outputs.is_fork != 'true' }}
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
@@ -220,9 +212,9 @@ jobs:
distribution: "temurin"
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
with:
gradle-version: 9.6.0
gradle-version: 9.6.1
- name: Install Task
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
@@ -240,12 +232,7 @@ jobs:
MAVEN_PUBLIC_URL: ${{ secrets.MAVEN_PUBLIC_URL }}
STIRLING_PDF_DESKTOP_UI: false
- name: Set up Depot CLI
if: env.USE_DEPOT == 'true'
uses: depot/setup-action@15c09a5f77a0840ad4bce955686522a257853461 # v1.0.0
- name: Set up Docker Buildx
if: env.USE_DEPOT != 'true'
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
- name: Login to Docker Hub
@@ -254,22 +241,7 @@ jobs:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_API }}
- name: Build and push PR-specific image (Depot)
if: env.USE_DEPOT == 'true'
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: .
file: ./docker/embedded/Dockerfile
push: true
tags: ${{ secrets.DOCKER_HUB_USERNAME }}/test:pr-${{ needs.check-comment.outputs.pr_number }}
build-args: |
VERSION_TAG=alpha
PROTOTYPES_BUILD=${{ needs.check-comment.outputs.enable_prototypes }}
platforms: linux/amd64
- name: Build and push PR-specific image (Docker fork fallback)
if: env.USE_DEPOT != 'true'
- name: Build and push PR-specific image
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
with:
context: .
@@ -283,19 +255,8 @@ jobs:
PROTOTYPES_BUILD=${{ needs.check-comment.outputs.enable_prototypes }}
platforms: linux/amd64
- name: Build and push engine image (Depot)
if: env.USE_DEPOT == 'true' && needs.check-comment.outputs.enable_prototypes == 'true'
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: ./engine
file: ./engine/Dockerfile
push: true
tags: ${{ secrets.DOCKER_HUB_USERNAME }}/test:engine-pr-${{ needs.check-comment.outputs.pr_number }}
platforms: linux/amd64
- name: Build and push engine image (Docker fork fallback)
if: env.USE_DEPOT != 'true' && needs.check-comment.outputs.enable_prototypes == 'true'
- name: Build and push engine image
if: needs.check-comment.outputs.enable_prototypes == 'true'
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
with:
context: ./engine
@@ -510,8 +471,7 @@ jobs:
handle-label-commands:
if: ${{ github.event.issue.pull_request != null }}
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
.github/workflows/_runner-pick.yml
+17 -20
View File
@@ -2,8 +2,8 @@ name: _runner-pick
# Tiny reusable workflow that classifies the trigger as either a "fork PR
# from an untrusted contributor" or a "trusted commit" so downstream jobs
# can pick a runner class without each one duplicating the 200-char gate
# expression in their own `runs-on:`.
# can trust-gate (skip secret-dependent jobs on forks) without each one
# duplicating the gate expression.
#
# Caller pattern:
#
@@ -13,12 +13,12 @@ name: _runner-pick
#
# real-work:
# needs: pick
# runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-8' }}
# if: needs.pick.outputs.is_fork != 'true'
# steps: [...]
#
# Output:
# is_fork: "true" when the trigger is a pull_request from a fork or an
# untrusted author_association, "false" otherwise.
# Outputs:
# is_fork: "true" when the trigger is a pull_request from a fork or an
# untrusted author_association, "false" otherwise.
on:
workflow_call:
@@ -50,21 +50,18 @@ jobs:
AUTHOR_ASSOC: ${{ github.event.pull_request.author_association }}
run: |
set -eu
if [ -z "${PR_NUMBER:-}" ]; then
# Not a pull_request event at all (push, schedule, workflow_dispatch,
# workflow_call from a non-PR trigger) -> trusted by default.
echo "is_fork=false" >> "$GITHUB_OUTPUT"
exit 0
is_fork=false
elif [ "${HEAD_REPO_FORK}" = "true" ]; then
is_fork=true
else
case "${AUTHOR_ASSOC}" in
OWNER|MEMBER|COLLABORATOR) is_fork=false ;;
*) is_fork=true ;;
esac
fi
if [ "${HEAD_REPO_FORK}" = "true" ]; then
echo "is_fork=true" >> "$GITHUB_OUTPUT"
exit 0
fi
case "${AUTHOR_ASSOC}" in
OWNER|MEMBER|COLLABORATOR)
echo "is_fork=false" >> "$GITHUB_OUTPUT"
;;
*)
echo "is_fork=true" >> "$GITHUB_OUTPUT"
;;
esac
echo "is_fork=${is_fork}" >> "$GITHUB_OUTPUT"
.github/workflows/ai-engine.yml
+1 -2
View File
@@ -18,8 +18,6 @@ jobs:
permissions:
contents: read
pull-requests: write
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
@@ -33,6 +31,7 @@ jobs:
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
with:
enable-cache: true
cache-suffix: ai-engine
- name: Install Task
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
.github/workflows/backend-build.yml
+4 -10
View File
@@ -19,14 +19,8 @@ permissions:
pull-requests: write
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
build:
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-8' }}
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
@@ -56,9 +50,9 @@ jobs:
key: gradle-deps-${{ runner.os }}-jdk-${{ matrix.jdk-version }}-${{ hashFiles('**/gradle/wrapper/gradle-wrapper.properties', '**/*.gradle', '**/*.gradle.kts', 'settings.gradle', 'settings.gradle.kts', 'gradle/libs.versions.toml') }}
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
with:
gradle-version: 9.6.0
gradle-version: 9.6.1
cache-disabled: true
- name: Install Task
@@ -247,7 +241,7 @@ jobs:
# so skip it for merge_group runs and workflow_dispatch.
if: github.event_name == 'pull_request'
id: jacoco
uses: madrapps/jacoco-report@50d3aff4548aa991e6753342d9ba291084e63848 # v1.7.2
uses: madrapps/jacoco-report@e51ce1f46f7f8b5331593f935e59cbaf44b84920 # v1.8.0
with:
paths: |
${{ github.workspace }}/**/build/reports/jacoco/test/jacocoTestReport.xml
.github/workflows/build-enterprise.yml
+4 -17
View File
@@ -15,23 +15,11 @@ name: Enterprise E2E (Playwright)
on:
workflow_call:
inputs:
depot_cores:
description: "Depot runner vCPU count (used in runs-on). Override for benchmarking."
required: false
type: string
default: "8"
push:
branches: ["main"]
schedule:
- cron: "0 4 * * *"
workflow_dispatch:
inputs:
depot_cores:
description: "Depot runner vCPU count (used in runs-on). Override for benchmarking."
required: false
type: string
default: "8"
# No `concurrency:` block here on purpose. When this workflow is called via
# workflow_call from build.yml, ${{ github.workflow }}/event_name/pr_number
@@ -50,17 +38,16 @@ jobs:
playwright-e2e-enterprise:
needs: pick
# Skip on fork PRs / untrusted authors: they have no PREMIUM_KEY_ENTERPRISE
# (nor DEPOT_TOKEN), so the suite can't boot premium and would fail. See the
# header comment. GitHub reports the skipped reusable workflow as success.
# Skip on fork PRs / untrusted authors: they have no PREMIUM_KEY_ENTERPRISE,
# so the suite can't boot premium and would fail. See the header comment.
# GitHub reports the skipped reusable workflow as success.
if: needs.pick.outputs.is_fork != 'true'
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || format('depot-ubuntu-24.04-{0}', inputs.depot_cores || '8') }}
runs-on: ubuntu-latest
timeout-minutes: 45
env:
PREMIUM_KEY: ${{ secrets.PREMIUM_KEY_ENTERPRISE }}
PREMIUM_ENABLED: "true"
SYSTEM_ENABLEANALYTICS: "false"
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
.github/workflows/build.yml
+20 -1
View File
@@ -41,6 +41,7 @@ jobs:
openapi: ${{ steps.changes.outputs.openapi }}
frontend: ${{ steps.changes.outputs.frontend }}
docker-base: ${{ steps.changes.outputs.docker-base }}
dockerfiles: ${{ steps.changes.outputs.dockerfiles }}
tauri: ${{ steps.changes.outputs.tauri }}
engine: ${{ steps.changes.outputs.engine }}
generated-models: ${{ steps.changes.outputs.generated-models }}
@@ -98,6 +99,17 @@ jobs:
uses: ./.github/workflows/frontend-validation.yml
secrets: inherit
# Advisory: deliberately NOT in all-checks-passed. It reports on the stories a
# branch touches so a regression is visible in review, but a browser scan is
# too new here to block merges on. Promote it once its pass/fail proves stable.
frontend-a11y:
if: needs.files-changed.outputs.frontend == 'true'
needs: [files-changed]
permissions:
contents: read
uses: ./.github/workflows/frontend-a11y.yml
secrets: inherit
playwright-e2e:
if: needs.files-changed.outputs.frontend == 'true'
needs: [files-changed]
@@ -148,11 +160,11 @@ jobs:
permissions:
contents: read
packages: read
id-token: write
uses: ./.github/workflows/test-build-docker.yml
secrets: inherit
with:
docker-base-changed: ${{ needs.files-changed.outputs.docker-base }}
dockerfiles-changed: ${{ needs.files-changed.outputs.dockerfiles }}
tauri-build:
if: needs.files-changed.outputs.tauri == 'true'
@@ -162,6 +174,13 @@ jobs:
pull-requests: write
uses: ./.github/workflows/tauri-build.yml
secrets: inherit
# PR smoke build: Linux only (fastest + cheapest to compile), unsigned,
# deb-only, no AppImage. The full signed multi-OS matrix runs on release;
# nightly still warms the Rust cache with all-OS defaults.
with:
platform: linux
sign: false
minimal: true
ai-engine:
if: needs.files-changed.outputs.engine == 'true'
.github/workflows/check-generated-models.yml
+2 -3
View File
@@ -21,8 +21,6 @@ jobs:
permissions:
contents: read
pull-requests: write
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
@@ -36,6 +34,7 @@ jobs:
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
with:
enable-cache: true
cache-suffix: generated-models
- name: Set up JDK 25
uses: actions/setup-java@be666c2fcd27ec809703dec50e508c2fdc7f6654 # v5.2.0
@@ -44,7 +43,7 @@ jobs:
distribution: "temurin"
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
with:
gradle-version: 9.6.0
.github/workflows/check-licence.yml
+2 -4
View File
@@ -11,8 +11,6 @@ permissions:
jobs:
check-licence:
runs-on: ubuntu-latest
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
@@ -38,9 +36,9 @@ jobs:
key: gradle-deps-${{ runner.os }}-jdk-25-${{ hashFiles('**/gradle/wrapper/gradle-wrapper.properties', '**/*.gradle', '**/*.gradle.kts', 'settings.gradle', 'settings.gradle.kts', 'gradle/libs.versions.toml') }}
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
with:
gradle-version: 9.6.0
gradle-version: 9.6.1
cache-disabled: true
- name: Install Task
.github/workflows/check-openapi.yml
+3 -9
View File
@@ -10,14 +10,8 @@ permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
check-generate-openapi-docs:
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
@@ -43,9 +37,9 @@ jobs:
key: gradle-deps-${{ runner.os }}-jdk-25-${{ hashFiles('**/gradle/wrapper/gradle-wrapper.properties', '**/*.gradle', '**/*.gradle.kts', 'settings.gradle', 'settings.gradle.kts', 'gradle/libs.versions.toml') }}
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
with:
gradle-version: 9.6.0
gradle-version: 9.6.1
cache-disabled: true
- name: Install Task
.github/workflows/check_toml.yml
+1 -1
View File
@@ -196,7 +196,7 @@ jobs:
core.exportVariable("REFERENCE_FILE", referenceFilePath);
- name: Set up Python
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.12"
.github/workflows/coverage-aggregate.yml
+4 -8
View File
@@ -29,12 +29,8 @@ permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
aggregate:
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- name: Harden Runner
@@ -60,13 +56,13 @@ jobs:
key: gradle-deps-${{ runner.os }}-jdk-25-${{ hashFiles('**/gradle/wrapper/gradle-wrapper.properties', '**/*.gradle', '**/*.gradle.kts', 'settings.gradle', 'settings.gradle.kts', 'gradle/libs.versions.toml') }}
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
with:
gradle-version: 9.6.0
gradle-version: 9.6.1
cache-disabled: true
- name: Set up Python
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.12"
.github/workflows/db-migration-test.yml
+3 -9
View File
@@ -12,15 +12,9 @@ permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
migration-test:
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-8' }}
runs-on: ubuntu-latest
timeout-minutes: 30
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
@@ -46,9 +40,9 @@ jobs:
key: gradle-deps-${{ runner.os }}-jdk-25-${{ hashFiles('**/gradle/wrapper/gradle-wrapper.properties', '**/*.gradle', '**/*.gradle.kts', 'settings.gradle', 'settings.gradle.kts', 'gradle/libs.versions.toml') }}
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
with:
gradle-version: 9.6.0
gradle-version: 9.6.1
cache-disabled: true
# No `-PnoSpotless` here yet because the upstream cache layer matches the
.github/workflows/deploy-on-v2-commit.yml
+5 -48
View File
@@ -10,21 +10,11 @@ permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
deploy-v2-on-push:
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
runs-on: ubuntu-latest
concurrency:
group: deploy-v2-push-V2
cancel-in-progress: true
permissions:
contents: read
id-token: write
env:
USE_DEPOT: ${{ needs.pick.outputs.is_fork != 'true' }}
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
@@ -35,12 +25,7 @@ jobs:
- name: Checkout code
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Set up Depot CLI
if: env.USE_DEPOT == 'true'
uses: depot/setup-action@15c09a5f77a0840ad4bce955686522a257853461 # v1.0.0
- name: Set up Docker Buildx
if: env.USE_DEPOT != 'true'
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
- name: Get commit hashes for frontend and backend
@@ -105,22 +90,8 @@ jobs:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_API }}
- name: Build and push frontend image (Depot)
if: env.USE_DEPOT == 'true' && steps.check-frontend.outputs.exists == 'false'
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: .
file: ./docker/frontend/Dockerfile
push: true
tags: |
${{ secrets.DOCKER_HUB_USERNAME }}/test:v2-frontend-${{ steps.commit-hashes.outputs.frontend_short }}
${{ secrets.DOCKER_HUB_USERNAME }}/test:v2-frontend-latest
build-args: VERSION_TAG=v2-alpha
platforms: linux/amd64
- name: Build and push frontend image (Docker fork fallback)
if: env.USE_DEPOT != 'true' && steps.check-frontend.outputs.exists == 'false'
- name: Build and push frontend image
if: steps.check-frontend.outputs.exists == 'false'
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
with:
context: .
@@ -134,22 +105,8 @@ jobs:
build-args: VERSION_TAG=v2-alpha
platforms: linux/amd64
- name: Build and push backend image (Depot)
if: env.USE_DEPOT == 'true' && steps.check-backend.outputs.exists == 'false'
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: .
file: ./docker/backend/Dockerfile
push: true
tags: |
${{ secrets.DOCKER_HUB_USERNAME }}/test:v2-backend-${{ steps.commit-hashes.outputs.backend_short }}
${{ secrets.DOCKER_HUB_USERNAME }}/test:v2-backend-latest
build-args: VERSION_TAG=v2-alpha
platforms: linux/amd64
- name: Build and push backend image (Docker fork fallback)
if: env.USE_DEPOT != 'true' && steps.check-backend.outputs.exists == 'false'
- name: Build and push backend image
if: steps.check-backend.outputs.exists == 'false'
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
with:
context: .
.github/workflows/docker-compose-tests.yml
+4 -15
View File
@@ -11,28 +11,17 @@ on:
required: false
type: string
default: "false"
depot_cores:
description: "Depot runner vCPU count (used in runs-on). Override for benchmarking. Tuned to 4 because bench showed 16 was within noise of 4."
required: false
type: string
default: "4"
permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
docker-compose-tests:
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || format('depot-ubuntu-24.04-{0}', inputs.depot_cores || '4') }}
runs-on: ubuntu-latest
permissions:
actions: write
contents: read
checks: write
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
@@ -59,9 +48,9 @@ jobs:
key: gradle-deps-${{ runner.os }}-jdk-25-${{ hashFiles('**/gradle/wrapper/gradle-wrapper.properties', '**/*.gradle', '**/*.gradle.kts', 'settings.gradle', 'settings.gradle.kts', 'gradle/libs.versions.toml') }}
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
with:
gradle-version: 9.6.0
gradle-version: 9.6.1
cache-disabled: true
# When the PR changes the base image, test.sh builds it locally
@@ -85,7 +74,7 @@ jobs:
sudo chmod +x /usr/local/bin/docker-compose
- name: Set up Python
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.12"
cache: "pip" # caching pip dependencies
.github/workflows/e2e-live.yml
+3 -13
View File
@@ -5,23 +5,13 @@ name: Playwright E2E (live backend)
# server.
on:
workflow_call:
inputs:
depot_cores:
description: "Depot runner vCPU count (used in runs-on). Override for benchmarking."
required: false
type: string
default: "8"
permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
playwright-e2e-live:
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || format('depot-ubuntu-24.04-{0}', inputs.depot_cores || '8') }}
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Harden Runner
@@ -94,7 +84,7 @@ jobs:
fi
- name: Set up Python for coverage summary
if: always() && steps.live-coverage.outputs.report == 'true'
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.12"
- name: Install defusedxml for coverage summary
@@ -134,7 +124,7 @@ jobs:
# a summary even on backend failure, as long as some Playwright
# tests ran far enough to dump V8 coverage.
if: always()
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.12"
.github/workflows/e2e-stubbed.yml
+1 -11
View File
@@ -5,23 +5,13 @@ name: Playwright E2E (stubbed)
# mocks API responses in the browser.
on:
workflow_call:
inputs:
depot_cores:
description: "Depot runner vCPU count (used in runs-on). Override for benchmarking. Tuned to 8 to match the other playwright workflows; bench showed flat scaling above 8."
required: false
type: string
default: "8"
permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
playwright-e2e:
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || format('depot-ubuntu-24.04-{0}', inputs.depot_cores || '8') }}
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
.github/workflows/frontend-a11y.yml
+57
View File
@@ -0,0 +1,57 @@
name: Frontend a11y regression gate
# Reusable workflow called from build.yml when frontend sources change.
#
# Scans the stories this branch touches in real Chromium and runs axe against
# each. Existing violations are grandfathered in .storybook/a11y-baseline.json;
# the check fails on a NEW violation — a story breaking a rule it wasn't already
# breaking — or on a story that fails to render at all.
#
# Only changed stories, because a full sweep is ~30 minutes: far too slow to sit
# in front of every merge. The whole suite is scanned nightly instead
# (nightly.yml), which catches anything a branch didn't touch.
#
# Advisory for now: this is not in build.yml's all-checks-passed list, so a
# failure reports without blocking. Promote it once a few weeks of runs show the
# pass/fail is stable.
on:
workflow_call:
permissions:
contents: read
jobs:
frontend-a11y:
runs-on: ubuntu-latest
timeout-minutes: 25
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
# Need the base branch too, to diff against it.
fetch-depth: 0
- name: Set up Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: "22"
cache: "npm"
cache-dependency-path: frontend/package-lock.json
- name: Install Task
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
- name: a11y gate (changed stories)
run: task frontend:storybook:a11y:changed -- origin/${{ github.base_ref || 'main' }}
- name: Upload scan reports
# The reports carry the offending selector and help text for each
# violation; without them a red run can only be understood by
# reproducing the whole scan locally.
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: a11y-scan-${{ github.run_id }}
path: frontend/.a11y-scan/
retention-days: 7
if-no-files-found: ignore
.github/workflows/frontend-backend-licenses-update.yml
+15 -15
View File
@@ -19,13 +19,9 @@ permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
files-changed:
name: detect what files changed
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
runs-on: ubuntu-latest
timeout-minutes: 3
outputs:
licenses-frontend: ${{ steps.changes.outputs.licenses-frontend }}
@@ -48,8 +44,8 @@ jobs:
generate-frontend-license-report:
if: needs.files-changed.outputs.licenses-frontend == 'true'
name: Generate Frontend License Report
needs: [pick, files-changed]
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
needs: files-changed
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
@@ -299,7 +295,10 @@ jobs:
base: main
title: "Update Frontend 3rd Party Licenses"
body: ${{ env.PR_BODY }}
labels: Licenses,github-actions,frontend
labels: |
Licenses
github-actions
Front End
draft: false
delete-branch: true
sign-commits: true
@@ -318,15 +317,13 @@ jobs:
generate-backend-license-report:
if: needs.files-changed.outputs.licenses-backend == 'true'
needs: [pick, files-changed]
needs: files-changed
name: Generate Backend License Report
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
repository-projects: write # Required for enabling automerge
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
@@ -354,9 +351,9 @@ jobs:
distribution: "temurin"
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
with:
gradle-version: 9.6.0
gradle-version: 9.6.1
- name: Install Task
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
@@ -520,7 +517,10 @@ jobs:
base: main
title: "Update Backend 3rd Party Licenses"
body: ${{ env.PR_BODY }}
labels: Licenses,github-actions,backend
labels: |
Licenses
github-actions
Back End
delete-branch: true
sign-commits: true
.github/workflows/frontend-validation.yml
+2 -6
View File
@@ -11,12 +11,8 @@ permissions:
pull-requests: write
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
frontend-validation:
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
@@ -121,7 +117,7 @@ jobs:
run: task frontend:test:coverage
- name: Set up Python for coverage summary
if: always()
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.12"
- name: Install defusedxml for coverage summary
.github/workflows/multiOSReleases.yml
+34 -24
View File
@@ -36,13 +36,9 @@ permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
determine-matrix:
if: ${{ vars.CI_PROFILE != 'lite' }}
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
runs-on: ubuntu-latest
outputs:
matrix: ${{ steps.set-matrix.outputs.matrix }}
version: ${{ steps.versionNumber.outputs.versionNumber }}
@@ -71,9 +67,9 @@ jobs:
gradle-${{ runner.os }}-
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
with:
gradle-version: 9.6.0
gradle-version: 9.6.1
- name: Install Task
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
@@ -112,10 +108,8 @@ jobs:
fi
build-jars:
needs: [pick, determine-matrix]
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
needs: determine-matrix
runs-on: ubuntu-latest
strategy:
matrix:
variant:
@@ -146,9 +140,9 @@ jobs:
distribution: "temurin"
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
with:
gradle-version: 9.6.0
gradle-version: 9.6.1
- name: Setup Node.js
if: matrix.variant.build_frontend == true
@@ -195,7 +189,6 @@ jobs:
SM_API_KEY: ${{ secrets.SM_API_KEY }}
WINDOWS_CERTIFICATE: ${{ secrets.WINDOWS_CERTIFICATE }}
RELEASE_GPG_PRIVATE_KEY: ${{ secrets.RELEASE_GPG_PRIVATE_KEY }}
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
@@ -250,9 +243,9 @@ jobs:
distribution: "temurin"
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
with:
gradle-version: 9.6.0
gradle-version: 9.6.1
- name: Install Task
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
@@ -510,6 +503,7 @@ jobs:
# cargo output unsigned, so checking it produces false negatives.
- name: Verify Windows Code Signature
if: ${{ matrix.platform == 'windows-latest' && env.SM_API_KEY != '' && (github.event_name == 'release' || (github.event_name == 'workflow_dispatch' && github.event.inputs.sign != 'false') || github.ref == 'refs/heads/V2-master') }}
timeout-minutes: 15
shell: pwsh
run: |
$allSigned = $true
@@ -531,11 +525,26 @@ jobs:
# Extract MSI and verify the inner exe (the file that actually gets installed).
# This is the critical check - AV flags the installed exe at runtime.
# Use lessmsi, not `msiexec /a`: msiexec serializes on the global
# _MSIExecute mutex and hangs forever on hosted runners when another
# installer is busy. lessmsi reads MSI tables directly - no mutex, no service.
$msi = $msiFiles[0].FullName
$extractDir = Join-Path $env:RUNNER_TEMP "msi-verify"
if (Test-Path $extractDir) { Remove-Item $extractDir -Recurse -Force }
$proc = Start-Process msiexec.exe -ArgumentList '/a', $msi, '/qn', "TARGETDIR=$extractDir" -Wait -PassThru -NoNewWindow
if ($proc.ExitCode -eq 0) {
New-Item -ItemType Directory -Force -Path $extractDir | Out-Null
choco install lessmsi -y --no-progress --limit-output | Out-Null
# Bound the extraction and kill on hang (defence in depth over timeout-minutes).
$proc = Start-Process lessmsi -ArgumentList 'x', "`"$msi`"", "`"$extractDir\`"" -PassThru -NoNewWindow
if (-not $proc.WaitForExit(120000)) {
try { $proc.Kill() } catch {}
Write-Host "[ERROR] MSI extraction timed out after 120s"
$allSigned = $false
} elseif ($proc.ExitCode -ne 0) {
Write-Host "[ERROR] Failed to extract MSI for verification (exit code: $($proc.ExitCode))"
$allSigned = $false
} else {
$innerExe = Get-ChildItem -Path $extractDir -Filter "stirling-pdf.exe" -Recurse -File | Select-Object -First 1
if ($innerExe) {
$sig = Get-AuthenticodeSignature -FilePath $innerExe.FullName
@@ -548,9 +557,6 @@ jobs:
Write-Host "[ERROR] Could not find stirling-pdf.exe inside MSI"
$allSigned = $false
}
} else {
Write-Host "[ERROR] Failed to extract MSI for verification (exit code: $($proc.ExitCode))"
$allSigned = $false
}
if (-not $allSigned) {
@@ -625,8 +631,8 @@ jobs:
retention-days: 1
collect-and-release:
needs: [pick, determine-matrix, build, build-jars]
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
needs: [determine-matrix, build, build-jars]
runs-on: ubuntu-latest
permissions:
contents: write
steps:
@@ -800,7 +806,11 @@ jobs:
uses: softprops/action-gh-release@b4309332981a82ec1c5618f44dd2e27cc8bfbfda # v3.0.0
with:
tag_name: v${{ needs.determine-matrix.outputs.version }}
generate_release_notes: true
# Don't regenerate/append notes on re-runs, and don't force this into the
# "Latest" slot - leave the release body and latest marker as they are.
generate_release_notes: false
append_body: false
make_latest: false
fail_on_unmatched_files: true
# Installers + updater payloads + manifest. .sig contents are embedded
# in latest.json so the .sig files themselves are not uploaded.
.github/workflows/nightly.yml
+41 -5
View File
@@ -13,13 +13,9 @@ permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
playwright-all-browsers:
name: Playwright (chromium + firefox + webkit)
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
runs-on: ubuntu-latest
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
@@ -57,6 +53,46 @@ jobs:
path: frontend/playwright-report/
retention-days: 14
# Whole-suite accessibility sweep. Pull requests only scan the stories they
# touch (frontend-a11y.yml) because a full pass takes ~30 minutes; this covers
# everything else, so a violation introduced by a change somewhere other than
# the story itself — a shared component, a theme token — still surfaces within
# a day.
a11y-all-stories:
name: a11y (every story)
runs-on: ubuntu-latest
timeout-minutes: 60
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Set up Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: "22"
cache: "npm"
cache-dependency-path: frontend/package-lock.json
- name: Install Task
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
- name: a11y gate (every story)
run: task frontend:storybook:a11y
- name: Upload scan reports
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: a11y-scan-nightly-${{ github.run_id }}
path: frontend/.a11y-scan/
retention-days: 14
if-no-files-found: ignore
# Builds all desktop platforms on a schedule so the Rust dependency cache is
# written on main, where PR and merge-queue tauri builds can restore it.
warm-tauri-cache:
.github/workflows/pr-conflict-labeler.yml
+159
View File
@@ -0,0 +1,159 @@
name: PR conflict labeler
on:
pull_request_target:
types:
- opened
- reopened
- synchronize
- edited
- ready_for_review
schedule:
- cron: "17 */6 * * *"
workflow_dispatch:
permissions:
contents: read
concurrency:
group: pr-conflict-labeler-${{ github.event.pull_request.number || 'all-open-prs' }}
cancel-in-progress: false
env:
CONFLICT_LABEL: "has conflicts"
jobs:
label-conflicts:
name: Label conflicted PRs
runs-on: ubuntu-latest
permissions:
contents: read
issues: write
pull-requests: read
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Check out the repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Set up stirling-bot token
id: setup-bot
uses: ./.github/actions/setup-bot
with:
app-id: ${{ secrets.GH_APP_ID }}
private-key: ${{ secrets.GH_APP_PRIVATE_KEY }}
- name: Apply conflict label
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
github-token: ${{ steps.setup-bot.outputs.token }}
script: |
const conflictLabel = process.env.CONFLICT_LABEL;
const owner = context.repo.owner;
const repo = context.repo.repo;
const eventPullRequest = context.payload.pull_request;
async function sleep(ms) {
await new Promise((resolve) => setTimeout(resolve, ms));
}
async function getPullRequestWithMergeableState(pullNumber) {
for (let attempt = 1; attempt <= 6; attempt += 1) {
const { data: pull } = await github.rest.pulls.get({
owner,
repo,
pull_number: pullNumber,
});
if (pull.mergeable !== null) {
return pull;
}
core.info(`PR #${pullNumber}: mergeable is not ready yet (attempt ${attempt}/6).`);
await sleep(5000);
}
const { data: pull } = await github.rest.pulls.get({
owner,
repo,
pull_number: pullNumber,
});
return pull;
}
async function ensureConflictLabel() {
try {
await github.rest.issues.getLabel({
owner,
repo,
name: conflictLabel,
});
} catch (error) {
if (error.status !== 404) {
throw error;
}
await github.rest.issues.createLabel({
owner,
repo,
name: conflictLabel,
color: 'D93F0B',
description: 'Pull request has merge conflicts with the base branch',
});
core.info(`Created '${conflictLabel}' label.`);
}
}
async function labelPullRequest(pull) {
const existingLabels = pull.labels.map((label) => label.name);
const hasConflictLabel = existingLabels.includes(conflictLabel);
const hasConflicts = pull.mergeable === false && pull.mergeable_state === 'dirty';
if (hasConflicts && !hasConflictLabel) {
await github.rest.issues.addLabels({
owner,
repo,
issue_number: pull.number,
labels: [conflictLabel],
});
core.info(`Added '${conflictLabel}' to PR #${pull.number}.`);
return;
}
if (!hasConflicts && hasConflictLabel) {
await github.rest.issues.removeLabel({
owner,
repo,
issue_number: pull.number,
name: conflictLabel,
});
core.info(`Removed '${conflictLabel}' from PR #${pull.number}.`);
return;
}
core.info(`PR #${pull.number}: no label change needed (mergeable=${pull.mergeable}, mergeable_state=${pull.mergeable_state}).`);
}
await ensureConflictLabel();
let pullNumbers;
if (eventPullRequest) {
pullNumbers = [eventPullRequest.number];
} else {
const pulls = await github.paginate(github.rest.pulls.list, {
owner,
repo,
state: 'open',
per_page: 100,
});
pullNumbers = pulls.map((pull) => pull.number);
core.info(`Checking ${pullNumbers.length} open PR(s).`);
}
for (const pullNumber of pullNumbers) {
const pull = await getPullRequestWithMergeableState(pullNumber);
await labelPullRequest(pull);
}
.github/workflows/pre_commit.yml
+1
View File
@@ -28,6 +28,7 @@ jobs:
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
with:
enable-cache: true
cache-suffix: pre-commit
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
.github/workflows/push-docker.yml
+2 -3
View File
@@ -76,9 +76,9 @@ jobs:
gradle-${{ runner.os }}-
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
with:
gradle-version: 9.6.0
gradle-version: 9.6.1
- name: Set up Docker Buildx
id: buildx
@@ -139,7 +139,6 @@ jobs:
tags: |
type=raw,value=${{ steps.versionNumber.outputs.versionNumber }},enable=${{ github.ref == 'refs/heads/master' || github.ref == 'refs/heads/V2-master' }}
type=raw,value=latest,enable=${{ github.ref == 'refs/heads/master' || github.ref == 'refs/heads/V2-master' }}
type=raw,value=alpha,enable=${{ github.ref == 'refs/heads/main' || github.ref == 'refs/heads/testMain' }}
- name: Build and push Unified Dockerfile (latest variant)
id: build-push-latest
.github/workflows/swagger.yml
+3 -9
View File
@@ -22,15 +22,9 @@ permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
push:
if: ${{ vars.CI_PROFILE != 'lite' }}
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
@@ -46,9 +40,9 @@ jobs:
distribution: "temurin"
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
with:
gradle-version: 9.6.0
gradle-version: 9.6.1
- name: Generate Swagger documentation
run: ./gradlew :stirling-pdf:generateOpenApiDocs
.github/workflows/sync-portal-docs.yml
+95
View File
@@ -0,0 +1,95 @@
name: Sync Portal Docs
# Regenerates the portal Developer Docs manifest from the Stirling docs repo and
# opens a PR when it changes. Runs weekly, on manual dispatch, or when the docs
# repo fires a `docs-updated` repository_dispatch.
on:
schedule:
- cron: "0 6 * * 1"
workflow_dispatch:
inputs:
ref:
description: "Docs repo ref (branch or tag) to sync from"
required: false
default: "main"
repository_dispatch:
types: [docs-updated]
concurrency:
group: ${{ github.workflow }}
cancel-in-progress: true
permissions:
contents: read
jobs:
sync:
name: Sync docs manifest
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: write
pull-requests: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
persist-credentials: false
- name: Setup GitHub App Bot
id: setup-bot
uses: ./.github/actions/setup-bot
with:
app-id: ${{ secrets.GH_APP_ID }}
private-key: ${{ secrets.GH_APP_PRIVATE_KEY }}
- name: Set up Node.js
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: "22"
cache: "npm"
cache-dependency-path: frontend/package-lock.json
- name: Install frontend dependencies
working-directory: frontend
env:
NPM_CONFIG_IGNORE_SCRIPTS: "true"
run: npm ci --ignore-scripts --audit=false --fund=false
- name: Regenerate docs manifest
working-directory: frontend
env:
DOCS_REF: ${{ github.event.inputs.ref || github.event.client_payload.ref || 'main' }}
GITHUB_TOKEN: ${{ steps.setup-bot.outputs.token }}
run: npm run docs:sync
- name: Create Pull Request
id: cpr
uses: peter-evans/create-pull-request@5f6978faf089d4d20b00c7766989d076bb2fc7f1 # v8.1.1
with:
token: ${{ steps.setup-bot.outputs.token }}
commit-message: "Sync portal docs from docs repo"
committer: ${{ steps.setup-bot.outputs.committer }}
author: ${{ steps.setup-bot.outputs.committer }}
signoff: true
branch: sync-portal-docs
base: main
title: "Sync portal docs from docs repo"
body: |
Auto-generated by ${{ steps.setup-bot.outputs.app-slug }}[bot].
Regenerates `frontend/editor/src/portal/generated/docsManifest.json`
from the Stirling docs repo via `npm run docs:sync`.
labels: |
Documentation
github-actions
Front End
add-paths: frontend/editor/src/portal/generated/docsManifest.json
delete-branch: true
sign-commits: true
.github/workflows/sync_files_v2.yml
+3 -1
View File
@@ -10,6 +10,7 @@ on:
- "app/common/build.gradle"
- "app/core/build.gradle"
- "app/proprietary/build.gradle"
- "gradle/spotless.gradle"
- "README.md"
- "frontend/editor/public/locales/*/translation.toml"
- "app/core/src/main/resources/static/3rdPartyLicenses.json"
@@ -51,7 +52,7 @@ jobs:
private-key: ${{ secrets.GH_APP_PRIVATE_KEY }}
- name: Set up Python
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.12"
cache: "pip" # caching pip dependencies
@@ -64,6 +65,7 @@ jobs:
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
with:
enable-cache: true
cache-suffix: sync-files
- name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
.github/workflows/tauri-build.yml
+22 -12
View File
@@ -21,6 +21,11 @@ on:
required: false
type: boolean
default: true
minimal:
description: "Fast smoke build: Linux deb only, skip rpm and the flaky AppImage pass. Used by PR builds."
required: false
type: boolean
default: false
workflow_dispatch:
inputs:
platform:
@@ -38,6 +43,11 @@ on:
required: false
default: true
type: boolean
minimal:
description: "Fast smoke build: Linux deb only, skip rpm and the flaky AppImage pass."
required: false
default: false
type: boolean
permissions:
contents: read
@@ -96,7 +106,6 @@ jobs:
WINDOWS_CERTIFICATE: ${{ secrets.WINDOWS_CERTIFICATE }}
APPLE_CERTIFICATE: ${{ secrets.APPLE_CERTIFICATE }}
RELEASE_GPG_PRIVATE_KEY: ${{ secrets.RELEASE_GPG_PRIVATE_KEY }}
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
@@ -158,9 +167,9 @@ jobs:
distribution: "temurin"
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
with:
gradle-version: 9.6.0
gradle-version: 9.6.1
- name: Setup Task
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
@@ -386,10 +395,10 @@ jobs:
with:
projectPath: ./frontend/editor
tauriScript: npx tauri
# Linux: build deb+rpm only here. AppImage runs in its own
# continue-on-error step below so its persistent linuxdeploy
# failure (#6127 onwards) does not tank deb/rpm uploads.
args: ${{ matrix.platform == 'ubuntu-22.04' && '--bundles deb,rpm' || matrix.args }}
# Linux: build deb+rpm only here (deb-only on minimal smoke builds).
# AppImage runs in its own continue-on-error step below so its
# persistent linuxdeploy failure (#6127 onwards) does not tank uploads.
args: ${{ matrix.platform == 'ubuntu-22.04' && (inputs.minimal && '--bundles deb' || '--bundles deb,rpm') || matrix.args }}
- name: Build Tauri app (unsigned)
if: ${{ !inputs.sign }}
@@ -406,15 +415,16 @@ jobs:
with:
projectPath: ./frontend/editor
tauriScript: npx tauri
# Linux: build deb+rpm only here. AppImage runs in its own
# continue-on-error step below so its persistent linuxdeploy
# failure (#6127 onwards) does not tank deb/rpm uploads.
args: ${{ matrix.platform == 'ubuntu-22.04' && '--bundles deb,rpm' || matrix.args }}
# Linux: build deb+rpm only here (deb-only on minimal smoke builds).
# AppImage runs in its own continue-on-error step below so its
# persistent linuxdeploy failure (#6127 onwards) does not tank uploads.
args: ${{ matrix.platform == 'ubuntu-22.04' && (inputs.minimal && '--bundles deb' || '--bundles deb,rpm') || matrix.args }}
# AppImage is decoupled so its linuxdeploy run gets a fresh process
# (rpm scratch state torn down) and its failure can't tank deb/rpm.
# Skipped on minimal smoke builds (flaky + slow, deb is enough to verify).
- name: Build Tauri app (Linux AppImage)
if: matrix.platform == 'ubuntu-22.04'
if: matrix.platform == 'ubuntu-22.04' && !inputs.minimal
continue-on-error: true
uses: tauri-apps/tauri-action@84b9d35b5fc46c1e45415bdb6144030364f7ebc5 # v0.6.2
env:
.github/workflows/test-build-docker.yml
+21 -70
View File
@@ -12,19 +12,16 @@ on:
required: false
type: string
default: "false"
depot_cores:
description: "Depot runner vCPU count (used in runs-on). Override for benchmarking."
dockerfiles-changed:
description: "Whether any Dockerfile changed (forwarded from files-changed). Gates the slow arm64 build leg."
required: false
type: string
default: "8"
default: "false"
permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
# TODO: extract a pre-matrix `prepare` job that runs once and produces
# shared artifacts for the three matrix entries below to consume:
# 1. `task backend:build` — currently runs 3× in parallel with
@@ -40,14 +37,7 @@ jobs:
# spring-security=true matrix entry if `task backend:build` and
# `task backend:build:ci` produce equivalent JARs (verify before wiring).
test-build-docker-images:
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || format('depot-ubuntu-24.04-{0}', inputs.depot_cores || '8') }}
permissions:
contents: read
id-token: write
env:
USE_DEPOT: ${{ needs.pick.outputs.is_fork != 'true' && inputs.docker-base-changed != 'true' }}
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
@@ -104,9 +94,9 @@ jobs:
key: gradle-deps-${{ runner.os }}-jdk-25-${{ hashFiles('**/gradle/wrapper/gradle-wrapper.properties', '**/*.gradle', '**/*.gradle.kts', 'settings.gradle', 'settings.gradle.kts', 'gradle/libs.versions.toml') }}
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
with:
gradle-version: 9.6.0
gradle-version: 9.6.1
cache-disabled: true
- name: Install Task
@@ -120,16 +110,10 @@ jobs:
DISABLE_ADDITIONAL_FEATURES: true
STIRLING_PDF_DESKTOP_UI: false
- name: Set up Depot CLI
if: env.USE_DEPOT == 'true'
uses: depot/setup-action@15c09a5f77a0840ad4bce955686522a257853461 # v1.0.0
- name: Set up QEMU
if: env.USE_DEPOT != 'true'
uses: docker/setup-qemu-action@ce360397dd3f832beb865e1373c09c0e9f86d70a # v4.0.0
- name: Set up Docker Buildx
if: env.USE_DEPOT != 'true'
id: buildx
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
@@ -146,13 +130,22 @@ jobs:
# GITHUB_EVENT_NAME is already provided by the runner.
env:
DOCKER_BASE_CHANGED: ${{ inputs.docker-base-changed }}
DOCKERFILES_CHANGED: ${{ inputs.dockerfiles-changed }}
run: |
if [ "$GITHUB_EVENT_NAME" = "pull_request" ] && [ "$DOCKER_BASE_CHANGED" = "true" ]; then
# Base Dockerfile changed: build against the locally-built base,
# which only exists for amd64.
echo "base_image=stirling-pdf-base:pr-test" >> "$GITHUB_OUTPUT"
echo "platforms=linux/amd64" >> "$GITHUB_OUTPUT"
else
elif [ "$DOCKERFILES_CHANGED" = "true" ]; then
# A Dockerfile changed: also verify the arm64 build (slow QEMU leg).
echo "base_image=stirlingtools/stirling-pdf-base:latest" >> "$GITHUB_OUTPUT"
echo "platforms=linux/amd64,linux/arm64/v8" >> "$GITHUB_OUTPUT"
else
# No Dockerfile change: amd64 only. arm64 is exercised on the base
# image publish and on release, not on every code PR.
echo "base_image=stirlingtools/stirling-pdf-base:latest" >> "$GITHUB_OUTPUT"
echo "platforms=linux/amd64" >> "$GITHUB_OUTPUT"
fi
# Base-changed PRs build the embedded image with the local docker driver
@@ -168,24 +161,10 @@ jobs:
--tag stirling-pdf-embedded:pr-test \
.
- name: Build ${{ matrix.docker-rev }} (Depot)
if: env.USE_DEPOT == 'true'
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: .
file: ./${{ matrix.docker-rev }}
push: false
platforms: ${{ steps.build-params.outputs.platforms }}
build-args: |
BASE_IMAGE=${{ steps.build-params.outputs.base_image }}
provenance: true
sbom: true
# Fork PRs that did NOT change the base use the buildx container builder
# PRs that did NOT change the base use the buildx container builder
# (multi-platform + gha cache) against the published base image.
- name: Build ${{ matrix.docker-rev }} (Docker fork fallback)
if: env.USE_DEPOT != 'true' && inputs.docker-base-changed != 'true'
- name: Build ${{ matrix.docker-rev }}
if: inputs.docker-base-changed != 'true'
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
with:
builder: ${{ steps.buildx.outputs.name }}
@@ -213,14 +192,7 @@ jobs:
if-no-files-found: warn
test-build-unoserver-image:
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || format('depot-ubuntu-24.04-{0}', inputs.depot_cores || '8') }}
permissions:
contents: read
id-token: write
env:
USE_DEPOT: ${{ needs.pick.outputs.is_fork != 'true' && inputs.docker-base-changed != 'true' }}
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
@@ -230,35 +202,14 @@ jobs:
- name: Checkout Repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Set up Depot CLI
if: env.USE_DEPOT == 'true'
uses: depot/setup-action@15c09a5f77a0840ad4bce955686522a257853461 # v1.0.0
- name: Set up QEMU
if: env.USE_DEPOT != 'true'
uses: docker/setup-qemu-action@ce360397dd3f832beb865e1373c09c0e9f86d70a # v4.0.0
- name: Set up Docker Buildx
if: env.USE_DEPOT != 'true'
id: buildx
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
- name: Build docker/unoserver/Dockerfile (Depot)
if: env.USE_DEPOT == 'true'
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: .
file: ./docker/unoserver/Dockerfile
push: false
load: true
platforms: linux/amd64
tags: stirling-unoserver:pr-test
provenance: false
sbom: false
- name: Build docker/unoserver/Dockerfile (Docker fork fallback)
if: env.USE_DEPOT != 'true'
- name: Build docker/unoserver/Dockerfile
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
with:
builder: ${{ steps.buildx.outputs.name }}
.github/workflows/testdriver.yml
+9 -38
View File
@@ -20,19 +20,9 @@ permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
deploy:
if: ${{ vars.CI_PROFILE != 'lite' }}
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
permissions:
contents: read
id-token: write
env:
USE_DEPOT: ${{ needs.pick.outputs.is_fork != 'true' }}
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
@@ -49,9 +39,9 @@ jobs:
distribution: "temurin"
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
with:
gradle-version: 9.6.0
gradle-version: 9.6.1
- name: Build with Gradle
run: ./gradlew build
@@ -61,12 +51,7 @@ jobs:
MAVEN_PUBLIC_URL: ${{ secrets.MAVEN_PUBLIC_URL }}
DISABLE_ADDITIONAL_FEATURES: true
- name: Set up Depot CLI
if: env.USE_DEPOT == 'true'
uses: depot/setup-action@15c09a5f77a0840ad4bce955686522a257853461 # v1.0.0
- name: Set up Docker Buildx
if: env.USE_DEPOT != 'true'
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
- name: Get version number
@@ -81,20 +66,7 @@ jobs:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_API }}
- name: Build and push test image (Depot)
if: env.USE_DEPOT == 'true'
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: .
file: ./docker/embedded/Dockerfile
push: true
tags: ${{ secrets.DOCKER_HUB_USERNAME }}/test:test-${{ github.sha }}
build-args: VERSION_TAG=${{ steps.versionNumber.outputs.versionNumber }}
platforms: linux/amd64
- name: Build and push test image (Docker fork fallback)
if: env.USE_DEPOT != 'true'
- name: Build and push test image
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
with:
context: .
@@ -153,8 +125,7 @@ jobs:
files-changed:
if: always()
name: detect what files changed
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
runs-on: ubuntu-latest
timeout-minutes: 3
outputs:
frontend: ${{ steps.changes.outputs.frontend }}
@@ -174,8 +145,8 @@ jobs:
test:
if: needs.files-changed.outputs.frontend == 'true'
needs: [pick, deploy, files-changed]
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
needs: [deploy, files-changed]
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
@@ -208,8 +179,8 @@ jobs:
FORCE_COLOR: "3"
cleanup:
needs: [pick, deploy, test]
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
needs: [deploy, test]
runs-on: ubuntu-latest
if: always()
steps:
.taskfiles/backend.yml
+2 -4
View File
@@ -26,7 +26,6 @@ tasks:
AIENGINE_ENABLED: '{{.AIENGINE_ENABLED}}'
AIENGINE_TIMEOUTSECONDS: '{{.AIENGINE_TIMEOUTSECONDS}}'
SECURITY_ENABLELOGIN: '{{.SECURITY_ENABLELOGIN}}'
POLICIES_ENABLED: '{{.POLICIES_ENABLED}}'
dev:proprietary:
desc: "Start backend dev server in proprietary mode"
@@ -41,13 +40,12 @@ tasks:
AIENGINE_ENABLED: '{{.AIENGINE_ENABLED | default "false"}}'
AIENGINE_TIMEOUTSECONDS: '{{.AIENGINE_TIMEOUTSECONDS | default "120"}}'
SECURITY_ENABLELOGIN: '{{.SECURITY_ENABLELOGIN | default ""}}'
POLICIES_ENABLED: '{{.POLICIES_ENABLED | default ""}}'
env:
SERVER_PORT: '{{.PORT}}'
cmds:
- cmd: '{{if .AIENGINE_URL}}AIENGINE_URL={{.AIENGINE_URL}} AIENGINE_ENABLED={{.AIENGINE_ENABLED}} AIENGINE_TIMEOUTSECONDS={{.AIENGINE_TIMEOUTSECONDS}} {{end}}{{if .SECURITY_ENABLELOGIN}}SECURITY_ENABLELOGIN={{.SECURITY_ENABLELOGIN}} {{end}}{{if .POLICIES_ENABLED}}POLICIES_ENABLED={{.POLICIES_ENABLED}} {{end}}cmd /c ".\gradlew.bat :stirling-pdf:bootRun"'
- cmd: '{{if .AIENGINE_URL}}AIENGINE_URL={{.AIENGINE_URL}} AIENGINE_ENABLED={{.AIENGINE_ENABLED}} AIENGINE_TIMEOUTSECONDS={{.AIENGINE_TIMEOUTSECONDS}} {{end}}{{if .SECURITY_ENABLELOGIN}}SECURITY_ENABLELOGIN={{.SECURITY_ENABLELOGIN}} {{end}}cmd /c ".\gradlew.bat :stirling-pdf:bootRun"'
platforms: [windows]
- cmd: '{{if .AIENGINE_URL}}AIENGINE_URL={{.AIENGINE_URL}} AIENGINE_ENABLED={{.AIENGINE_ENABLED}} AIENGINE_TIMEOUTSECONDS={{.AIENGINE_TIMEOUTSECONDS}} {{end}}{{if .SECURITY_ENABLELOGIN}}SECURITY_ENABLELOGIN={{.SECURITY_ENABLELOGIN}} {{end}}{{if .POLICIES_ENABLED}}POLICIES_ENABLED={{.POLICIES_ENABLED}} {{end}}./gradlew :stirling-pdf:bootRun'
- cmd: '{{if .AIENGINE_URL}}AIENGINE_URL={{.AIENGINE_URL}} AIENGINE_ENABLED={{.AIENGINE_ENABLED}} AIENGINE_TIMEOUTSECONDS={{.AIENGINE_TIMEOUTSECONDS}} {{end}}{{if .SECURITY_ENABLELOGIN}}SECURITY_ENABLELOGIN={{.SECURITY_ENABLELOGIN}} {{end}}./gradlew :stirling-pdf:bootRun'
platforms: [linux, darwin]
dev:bundled:
.taskfiles/frontend.yml
+78 -4
View File
@@ -85,7 +85,8 @@ tasks:
# full path, hostname, or user. Consumed at dev-serve time by vite.config
# and dropped from production builds.
STIRLING_DEV_LABEL:
sh: basename "$(git rev-parse --show-toplevel 2>/dev/null || pwd)"
sh: >-
{{if eq OS "windows"}}powershell -NoProfile -Command '$root = git rev-parse --show-toplevel 2>$null; if (-not $root) { $root = (Get-Location).Path }; Split-Path -Leaf $root'{{else}}basename "$(git rev-parse --show-toplevel 2>/dev/null || pwd)"{{end}}
cmds:
- npx vite editor --mode {{.MODE}} --port {{.PORT}}{{if .OPEN}} --open{{end}}
@@ -193,6 +194,64 @@ tasks:
cmds:
- npx storybook build {{.CLI_ARGS}}
storybook:browser:
internal: true
desc: "Install the Chromium build the story scan runs in"
run: once
deps: [install]
cmds:
- npx playwright install chromium
storybook:test:
desc: "Scan every story in real Chromium: it must render and pass axe"
deps: [install, storybook:browser]
cmds:
# Runs each story as a browser test. Pass a filter through, e.g.
# task frontend:storybook:test -- Button
- npx vitest run --config .storybook/vitest.config.ts {{.CLI_ARGS}}
storybook:a11y:
desc: "a11y regression gate over every story: fail only on NEW axe violations"
deps: [install, storybook:browser]
cmds:
- bash .storybook/a11y-scan.sh
- node .storybook/a11y-check.mjs --in .a11y-scan --manifest .a11y-scan/manifest.txt
storybook:a11y:changed:
desc: "a11y gate over stories changed vs a base ref (default origin/main)"
summary: |
Scans only the stories this branch touches, which is what pull requests
run — a full scan takes ~30 minutes, far too long to sit in front of every
merge. The nightly job covers the rest of the suite.
Pass a base ref through CLI_ARGS, e.g.
task frontend:storybook:a11y:changed -- origin/release
deps: [install, storybook:browser]
vars:
BASE: '{{.CLI_ARGS | default "origin/main"}}'
# Stories touched by this branch, plus any not yet committed.
CHANGED:
sh: |
{ git diff --name-only --diff-filter=d {{.CLI_ARGS | default "origin/main"}}...HEAD -- '*.stories.ts' '*.stories.tsx';
git diff --name-only --diff-filter=d -- '*.stories.ts' '*.stories.tsx';
git ls-files --others --exclude-standard -- '*.stories.ts' '*.stories.tsx'; } \
| sed 's|^frontend/||' | sort -u | tr '\n' ' '
cmds:
- cmd: |
if [ -z "{{.CHANGED}}" ]; then
echo "a11y: no story files changed vs {{.BASE}} — nothing to check"
exit 0
fi
bash .storybook/a11y-scan.sh {{.CHANGED}}
node .storybook/a11y-check.mjs --in .a11y-scan --manifest .a11y-scan/manifest.txt
storybook:a11y:record:
desc: "Re-record the a11y baseline (run after intentionally fixing/adding violations)"
deps: [install, storybook:browser]
cmds:
- bash .storybook/a11y-scan.sh
- node .storybook/a11y-check.mjs --in .a11y-scan --manifest .a11y-scan/manifest.txt --record
# ============================================================
# Code quality
# ============================================================
@@ -203,6 +262,23 @@ tasks:
cmds:
- task: lint:eslint
- task: lint:dpdm
- task: lint:colors
lint:colors:
desc: "Enforce theme tokens — no hardcoded colours or raw primitives in components"
aliases: [lint:colours]
deps: [install]
cmds:
- node editor/scripts/lint/theme-lint.mjs
- node editor/scripts/lint/theme-lint.mjs css-colors
- node editor/scripts/lint/theme-lint.mjs code-colors
- node editor/scripts/lint/theme-lint.mjs no-primitives
contrast:
desc: "Report low-contrast theme token pairs (warning only, never blocks)"
deps: [install]
cmds:
- node editor/scripts/lint/theme-lint.mjs contrast
lint:eslint:
desc: "Run ESLint linting"
@@ -249,10 +325,8 @@ tasks:
typecheck:_run:
internal: true
env:
CI: '{{ .CI | default "false" }}'
cmds:
- '{{ if eq .CI "true" }}npx tsc{{ else }}npx tsgo{{ end }} --noEmit --project {{.PROJECT}}'
- 'npx tsc --noEmit --project {{.PROJECT}}'
typecheck:core:
desc: "Typecheck core build variant"
AGENTS.md
+2
View File
@@ -155,6 +155,8 @@ The project structure is defined in `engine/pyproject.toml`. Any new dependencie
For a broader explanation of the frontend layering and override architecture, read @frontend/editor/DeveloperGuide.md
Before touching colours or theming (tokens, dark mode, accent colours), read @frontend/editor/src/core/theme/README.md — it explains the palette/`--c-*` token system and the rule that literal colours live only in `primitives.css`.
```typescript
// ✅ CORRECT - Use @app/* for all imports
import { AppLayout } from "@app/components/AppLayout";
LICENSE
+2
View File
@@ -22,6 +22,8 @@ if that directory exists, is licensed under the license defined in "frontend/edi
if that directory exists, is licensed under the license defined in "frontend/editor/src/prototypes/LICENSE".
* All content that resides under the "frontend/editor/src/portal/" directory of this repository,
if that directory exists, is licensed under the license defined in "frontend/editor/src/portal/LICENSE".
* All content that resides under the "frontend/editor/src/portal-saas/" directory of this repository,
if that directory exists, is licensed under the license defined in "frontend/editor/src/portal-saas/LICENSE".
* Content outside of the above mentioned directories or restrictions above is
available under the MIT License as defined below.
Taskfile.yml
-1
View File
@@ -90,7 +90,6 @@ tasks:
vars:
PORT: '{{.BACKEND_PORT}}'
SECURITY_ENABLELOGIN: "true"
POLICIES_ENABLED: "true"
- task: frontend:dev:proprietary
vars:
PORT: '{{.EDITOR_PORT}}'
app/allowed-licenses.json
+12
View File
@@ -208,6 +208,18 @@
"moduleName": ".*",
"moduleLicense": "The W3C License"
},
{
"moduleName": "com.google.re2j:re2j",
"moduleLicense": "Go License"
},
{
"moduleName": "com.hubspot:algebra",
"moduleLicense": null
},
{
"moduleName": "com.hubspot.immutables:immutables-exceptions",
"moduleLicense": null
},
{
"moduleName": ".*",
"moduleLicense": "UnRar License"
app/common/build.gradle
+1 -27
View File
@@ -2,32 +2,6 @@
bootRun {
enabled = false
}
spotless {
java {
target 'src/**/java/**/*.java'
targetExclude 'src/main/java/org/apache/**'
googleJavaFormat(googleJavaFormatVersion).aosp().reorderImports(false)
// google-java-format 1.28.0 bundles Guava 32.x which crashes Spotless lint on JDK 24/25
suppressLintsFor { setStep('google-java-format') }
importOrder("java", "javax", "org", "com", "net", "io", "jakarta", "lombok", "me", "stirling")
trimTrailingWhitespace()
leadingTabsToSpaces()
endWithNewline()
}
yaml {
target '**/*.yml', '**/*.yaml'
trimTrailingWhitespace()
leadingTabsToSpaces()
endWithNewline()
}
format 'gradle', {
target '**/gradle/*.gradle', '**/*.gradle'
trimTrailingWhitespace()
leadingTabsToSpaces()
endWithNewline()
}
}
dependencies {
api "com.google.guava:guava:${guavaVersion}"
api 'org.springframework.boot:spring-boot-starter-webmvc'
@@ -42,7 +16,7 @@ dependencies {
api "org.apache.pdfbox:pdfbox-io:$pdfboxVersion"
api "org.apache.pdfbox:xmpbox:$pdfboxVersion"
api "org.apache.pdfbox:preflight:$pdfboxVersion"
api 'com.github.junrar:junrar:7.5.10' // RAR archive support for CBR files
api 'com.github.junrar:junrar:7.6.0' // RAR archive support for CBR files
api 'jakarta.servlet:jakarta.servlet-api:6.1.0'
api 'org.snakeyaml:snakeyaml-engine:3.0.1'
api "org.springdoc:springdoc-openapi-starter-webmvc-ui:3.0.3"
app/common/src/main/java/stirling/software/common/model/ApplicationProperties.java
+123 -8
View File
@@ -207,16 +207,12 @@ public class ApplicationProperties {
@Data
public static class Policies {
/**
* Master switch for the policy + sources subsystem (the PAYG-metered automation surface).
*/
private boolean enabled = false;
/**
* Absolute directories that policy folder input sources and output sinks may read from or
* write to. Empty (the default) disables folder access entirely, so a policy can never be
* pointed at an arbitrary server path. Stirling's own config directory is always
* off-limits, and folder access is always disabled in SaaS mode regardless of this list.
* write to. Empty (the default) disables folder access except to implicitly defined
* folders, such as server storage folders (if enabled) and the pipeline watched folders.
* Stirling's own config directory is always off-limits, and folder access is always
* disabled in SaaS mode regardless of this list.
*/
private List<String> allowedFolderRoots = new java.util.ArrayList<>();
@@ -255,6 +251,29 @@ public class ApplicationProperties {
* in-network object store.
*/
private boolean allowPrivateS3Endpoints = false;
/**
* Whether an API/Purview/ConsignO integration's base URL may resolve to a loopback,
* link-local, or private address. Off by default: unlike S3 connections, any user may
* create one of these, so without this gate a user could point a connection at the cloud
* metadata address and have the server fetch it for them. Enable only when integrations
* genuinely live inside the network (e.g. an on-prem ConsignO or an internal API gateway).
*/
private boolean allowPrivateApiEndpoints = false;
/**
* Whether administrators may define their own API integrations - a free-form base URL,
* path, body and headers - as opposed to only using the built-in vendor presets (Purview,
* ConsignO, S3). On by default, and admin-only regardless: a custom integration can point
* the server at any host, so it is authoring power, not self-serve.
*
* <p>Turning this off stops new custom integrations being created or edited. Ones that
* already exist keep running, because a policy that silently stopped calling out would be a
* worse surprise than one that keeps working; disable the connection itself to stop it.
*/
private boolean allowCustomApiIntegrations = true;
private long webhookMaxBytes = 104857600L;
}
@Data
@@ -309,6 +328,102 @@ public class ApplicationProperties {
* explicitly requests it via {@code AiEngineClient.postWithTimeout}.
*/
private int longRunningTimeoutSeconds = 600;
/** Timeout (seconds) for the SSE stream held open by long-running orchestrator runs. */
private int streamTimeoutSeconds = 1800;
/**
* Whether the processor pushes settings-derived AI config to the engine on startup/save.
* Pin false for env-driven deployments (SaaS) to keep the engine env-controlled.
*/
private boolean pushConfigToEngine = true;
/** Model + provider selection, forwarded to the engine per-request. */
private Models models = new Models();
/** Retrieval-augmented-generation (RAG) knobs, forwarded to the engine per-request. */
private Rag rag = new Rag();
/** Request size / cost guardrails. */
private Limits limits = new Limits();
/** Per-capability on/off switches so an admin can disable individual AI tools. */
private Features features = new Features();
@Data
public static class Models {
/** Provider driving the model strings: 'anthropic', 'openai', 'ollama', or 'custom'. */
private String provider = "anthropic";
/** High-quality tier model name (without provider prefix), e.g. 'claude-haiku-4-5'. */
private String smartModel = "claude-haiku-4-5";
/** Cheap/fast tier model name (without provider prefix). */
private String fastModel = "claude-haiku-4-5";
private int smartMaxTokens = 8192;
private int fastMaxTokens = 2048;
/**
* API key for the selected provider (secret; masked). Empty means the engine uses its
* own env credential (e.g. ANTHROPIC_API_KEY).
*/
private String apiKey = "";
/**
* OpenAI-compatible base URL for 'ollama' / 'custom' providers (e.g.
* http://ollama:11434/v1). Ignored for anthropic/openai. SSRF-sensitive - admin only.
*/
private String baseUrl = "";
}
@Data
public static class Rag {
/**
* Embedding provider: 'voyageai', 'openai', 'ollama', or 'custom' (OpenAI-compatible).
*/
private String embeddingProvider = "voyageai";
/** Embedding model name (without provider prefix), e.g. 'voyage-4'. */
private String embeddingModel = "voyage-4";
/**
* Secret API key for the embedding provider; masked + env-overridable like
* models.apiKey.
*/
private String embeddingApiKey = "";
/**
* OpenAI-compatible base URL for 'ollama' / 'custom' embedding providers (e.g.
* http://ollama:11434/v1). Ignored for voyageai/openai. SSRF-sensitive - admin only.
*/
private String embeddingBaseUrl = "";
/** How many chunks retrieval returns per search. */
private int topK = 20;
/** Per-run cap on knowledge-search tool calls before the agent must answer. */
private int maxSearches = 5;
}
@Data
public static class Limits {
private int maxPages = 200;
private int maxCharacters = 200000;
/** Process-wide cap on concurrent model API calls (engine restart to apply). */
private int modelMaxConcurrency = 32;
}
@Data
public static class Features {
private boolean chat = true;
private boolean documentQuestions = true;
private boolean createPdf = true;
private boolean mathAuditor = true;
private boolean pdfComment = true;
private boolean classify = true;
}
}
/**
app/common/src/main/java/stirling/software/common/model/job/JobResult.java
+17
View File
@@ -4,6 +4,8 @@ import java.time.LocalDateTime;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.CopyOnWriteArrayList;
import com.fasterxml.jackson.annotation.JsonIgnore;
@@ -47,6 +49,9 @@ public class JobResult {
*/
private final List<String> notes = new CopyOnWriteArrayList<>();
/** Key/value metadata that survives the write-through into the shared job store. */
private final Map<String, String> metadata = new ConcurrentHashMap<>();
/**
* Create a new JobResult with the given job ID
*
@@ -161,4 +166,16 @@ public class JobResult {
public List<String> getNotes() {
return Collections.unmodifiableList(notes);
}
/** Attach a metadata value, e.g. a policy id so cluster peers can identify a policy run. */
public void putMetadata(String key, String value) {
if (key != null && value != null) {
this.metadata.put(key, value);
}
}
/** An unmodifiable view of this job's metadata. */
public Map<String, String> getMetadata() {
return Collections.unmodifiableMap(metadata);
}
}
app/common/src/main/java/stirling/software/common/service/InternalApiClient.java
+6 -1
View File
@@ -46,9 +46,14 @@ public class InternalApiClient {
// The second alternation carves out `/api/v1/ai/tools/*` specifically — AI tools are
// dispatchable, but the broader `/api/v1/ai/` surface (orchestrate, health, etc.) is
// intentionally NOT permitted to avoid plan steps re-entering the orchestrator.
//
// `/api/v1/integration/*` holds third-party steps (external API call, Purview labelling,
// ConsignO). They reach outside the JVM, so the namespace is deliberately kept to tools that
// dereference an admin-owned connection rather than a caller-supplied host — see
// ApiConnectionResolver.
private static final Pattern ALLOWED_ENDPOINT_PATH =
Pattern.compile(
"^/api/v1/(general|misc|security|convert|filter)(/[A-Za-z0-9_-]+)+$"
"^/api/v1/(general|misc|security|convert|filter|integration)(/[A-Za-z0-9_-]+)+$"
+ "|^/api/v1/ai/tools(/[A-Za-z0-9_-]+)+$");
/**
app/common/src/main/java/stirling/software/common/service/TaskManager.java
+13 -1
View File
@@ -230,6 +230,18 @@ public class TaskManager {
return false;
}
/** Attach metadata to a job and write it through to the shared store for cluster peers. */
public boolean putMetadata(String jobId, String key, String value) {
JobResult jobResult = jobResults.get(jobId);
if (jobResult != null) {
jobResult.putMetadata(key, value);
writeThrough(jobId, jobResult);
return true;
}
log.warn("Attempted to set metadata on non-existent job ID: {}", jobId);
return false;
}
/**
* Get statistics about all jobs in the system
*
@@ -378,7 +390,7 @@ public class TaskManager {
fileIds.add(rf.getFileId());
}
}
Map<String, String> meta = new HashMap<>();
Map<String, String> meta = new HashMap<>(result.getMetadata());
if (result.getNotes() != null && !result.getNotes().isEmpty()) {
meta.put("notesCount", Integer.toString(result.getNotes().size()));
}
app/common/src/main/java/stirling/software/common/util/PdfTextLocator.java
+4 -1
View File
@@ -5,6 +5,7 @@ import java.util.ArrayList;
import java.util.List;
import java.util.Locale;
import java.util.Optional;
import java.util.regex.Pattern;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.common.PDRectangle;
@@ -28,6 +29,8 @@ import lombok.extern.slf4j.Slf4j;
@Component
public class PdfTextLocator {
private static final Pattern NON_ALPHANUMERIC_PATTERN = Pattern.compile("[^A-Za-z0-9]");
/** One found line of text with its user-space bounding box. */
public record MatchedBox(float x, float y, float width, float height) {}
@@ -82,7 +85,7 @@ public class PdfTextLocator {
/** Strip everything non-alphanumeric and lowercase for tolerant matching. */
private static String normalize(String s) {
return s.replaceAll("[^A-Za-z0-9]", "").toLowerCase(Locale.ROOT);
return NON_ALPHANUMERIC_PATTERN.matcher(s).replaceAll("").toLowerCase(Locale.ROOT);
}
private static final class CapturedLine {
app/common/src/main/java/stirling/software/common/util/RequestUriUtils.java
+6 -1
View File
@@ -1,7 +1,11 @@
package stirling.software.common.util;
import java.util.regex.Pattern;
public class RequestUriUtils {
private static final Pattern SHARE_LINK_PATTERN = Pattern.compile("^/share/[^/]+/?$");
public static boolean isStaticResource(String requestURI) {
return isStaticResource("", requestURI);
}
@@ -198,11 +202,12 @@ public class RequestUriUtils {
|| trimmedUri.startsWith("/readiness")
|| trimmedUri.startsWith(
"/api/v1/mobile-scanner/") // Mobile scanner endpoints (no auth)
|| trimmedUri.startsWith("/api/v1/webhooks/")
|| trimmedUri.startsWith("/v1/api-docs")
// Workflow participant endpoints - access controlled by share tokens, not login
|| trimmedUri.startsWith("/api/v1/workflow/participant/")
// Share-link SPA bootstrap; data APIs remain protected
|| trimmedUri.matches("^/share/[^/]+/?$");
|| SHARE_LINK_PATTERN.matcher(trimmedUri).matches();
}
private static String stripContextPath(String contextPath, String requestURI) {
app/common/src/test/java/stirling/software/common/util/RequestUriUtilsTest.java
+7
View File
@@ -176,6 +176,13 @@ class RequestUriUtilsTest {
assertFalse(RequestUriUtils.isPublicAuthEndpoint("/api/v1/convert", ""));
}
@Test
void testIsPublicAuthEndpoint_webhookReceiver() {
// The webhook source receiver authenticates each delivery by HMAC signature, not a session.
assertTrue(RequestUriUtils.isPublicAuthEndpoint("/api/v1/webhooks/whk_abc123", ""));
assertTrue(RequestUriUtils.isPublicAuthEndpoint("/app/api/v1/webhooks/whk_abc123", "/app"));
}
@Test
void testIsPublicAuthEndpoint_withContextPath() {
assertTrue(RequestUriUtils.isPublicAuthEndpoint("/app/login", "/app"));
app/core/build.gradle
-29
View File
@@ -9,35 +9,6 @@ configurations {
}
}
spotless {
java {
target 'src/**/java/**/*.java'
targetExclude 'src/main/resources/static/**', 'src/main/java/org/apache/**'
googleJavaFormat(googleJavaFormatVersion).aosp().reorderImports(false)
// google-java-format 1.28.0 bundles Guava 32.x which crashes Spotless lint on JDK 24/25
suppressLintsFor { setStep('google-java-format') }
importOrder("java", "javax", "org", "com", "net", "io", "jakarta", "lombok", "me", "stirling")
trimTrailingWhitespace()
leadingTabsToSpaces()
endWithNewline()
}
yaml {
target '**/*.yml', '**/*.yaml'
targetExclude 'src/main/resources/static/**'
trimTrailingWhitespace()
leadingTabsToSpaces()
endWithNewline()
}
format 'gradle', {
target '**/gradle/*.gradle', '**/*.gradle'
targetExclude 'src/main/resources/static/**'
trimTrailingWhitespace()
leadingTabsToSpaces()
endWithNewline()
}
}
dependencies {
if (!gradle.ext.disableAdditional) {
implementation project(':proprietary')
app/core/src/main/java/stirling/software/SPDF/controller/api/form/FormPayloadParser.java
+8 -8
View File
@@ -117,8 +117,8 @@ final class FormPayloadParser {
names.add(single);
}
}
} else if (root.isTextual()) {
final String single = trimToNull(root.asText(""));
} else if (root.isString()) {
final String single = trimToNull(root.asString(""));
if (single != null) {
names.add(single);
}
@@ -197,8 +197,8 @@ final class FormPayloadParser {
if (node == null || node.isNull()) {
return null;
}
if (node.isTextual()) {
return trimToEmpty(node.asText(""));
if (node.isString()) {
return trimToEmpty(node.asString(""));
}
if (node.isNumber()) {
return node.numberValue().toString();
@@ -207,7 +207,7 @@ final class FormPayloadParser {
return Boolean.toString(node.booleanValue());
}
// Fallback for other scalar-like nodes
return trimToEmpty(node.asText(""));
return trimToEmpty(node.asString(""));
}
private static void collectNames(JsonNode arrayNode, Set<String> sink) {
@@ -227,8 +227,8 @@ final class FormPayloadParser {
return null;
}
if (node.isTextual()) {
return trimToNull(node.asText(""));
if (node.isString()) {
return trimToNull(node.asString(""));
}
if (node.isObject()) {
@@ -269,7 +269,7 @@ final class FormPayloadParser {
final JsonNode v = objectNode.get(key);
if (v == null || v.isNull()) {
result.put(key, null);
} else if (v.isTextual() || v.isNumber() || v.isBoolean()) {
} else if (v.isString() || v.isNumber() || v.isBoolean()) {
result.put(key, coerceScalarToString(v));
} else {
result.put(key, v.toString());
app/core/src/main/java/stirling/software/SPDF/controller/api/misc/ConfigController.java
+13 -1
View File
@@ -336,7 +336,19 @@ public class ConfigController {
configData.put("premiumEnabled", applicationProperties.getPremium().isEnabled());
// AI Engine settings
configData.put("aiEngineEnabled", applicationProperties.getAiEngine().isEnabled());
ApplicationProperties.AiEngine aiEngineConfig = applicationProperties.getAiEngine();
configData.put("aiEngineEnabled", aiEngineConfig.isEnabled());
// Per-capability flags let the UI hide individual AI tools an admin has turned off.
ApplicationProperties.AiEngine.Features aiFeatures = aiEngineConfig.getFeatures();
configData.put(
"aiFeatures",
Map.ofEntries(
Map.entry("chat", aiFeatures.isChat()),
Map.entry("documentQuestions", aiFeatures.isDocumentQuestions()),
Map.entry("createPdf", aiFeatures.isCreatePdf()),
Map.entry("mathAuditor", aiFeatures.isMathAuditor()),
Map.entry("pdfComment", aiFeatures.isPdfComment()),
Map.entry("classify", aiFeatures.isClassify())));
// Timestamp TSA settings — single source of truth for presets + admin URLs
ApplicationProperties.Security.Timestamp tsConfig =
app/core/src/main/java/stirling/software/SPDF/controller/api/security/ValidateSignatureController.java
+66 -6
View File
@@ -23,6 +23,9 @@ import org.bouncycastle.cms.CMSSignedData;
import org.bouncycastle.cms.SignerInformation;
import org.bouncycastle.cms.SignerInformationStore;
import org.bouncycastle.cms.jcajce.JcaSimpleSignerInfoVerifierBuilder;
import org.bouncycastle.operator.jcajce.JcaDigestCalculatorProviderBuilder;
import org.bouncycastle.tsp.TimeStampToken;
import org.bouncycastle.tsp.TimeStampTokenInfo;
import org.bouncycastle.util.Store;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
@@ -54,6 +57,9 @@ public class ValidateSignatureController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final CertificateValidationService certValidationService;
/** PDF sub-filter identifying an RFC 3161 document timestamp (PAdES-LTV). */
private static final String SUBFILTER_RFC3161 = "ETSI.RFC3161";
@InitBinder
public void initBinder(WebDataBinder binder) {
binder.registerCustomEditor(
@@ -128,8 +134,35 @@ public class ValidateSignatureController {
byte[] signedContent = sig.getSignedContent(file.getInputStream());
byte[] signatureBytes = sig.getContents(file.getInputStream());
CMSProcessable content = new CMSProcessableByteArray(signedContent);
CMSSignedData signedData = new CMSSignedData(content, signatureBytes);
// An RFC 3161 document timestamp (PAdES-LTV) carries its signed content
// *inside* the CMS - a TSTInfo - rather than being detached over the document.
// Building it as detached digests the ByteRange against an attribute that
// covers the TSTInfo, which can never match.
boolean isDocTimeStamp = SUBFILTER_RFC3161.equals(sig.getSubFilter());
CMSSignedData signedData;
if (isDocTimeStamp) {
signedData = new CMSSignedData(signatureBytes);
} else {
CMSProcessable content = new CMSProcessableByteArray(signedContent);
signedData = new CMSSignedData(content, signatureBytes);
}
// What actually binds a timestamp to this document: the TSTInfo's message
// imprint must equal the digest of the signed byte range. Without this check a
// valid timestamp token for some *other* document would verify happily here.
Date timeStampGenTime = null;
if (isDocTimeStamp) {
TimeStampToken token = new TimeStampToken(signedData);
TimeStampTokenInfo info = token.getTimeStampInfo();
timeStampGenTime = info.getGenTime();
if (!timestampCoversContent(info, signedContent)) {
result.setValid(false);
result.setErrorMessage(
"Timestamp message imprint does not match the document");
results.add(result);
continue;
}
}
Store<X509CertificateHolder> certStore = signedData.getCertificates();
SignerInformationStore signerStore = signedData.getSignerInfos();
@@ -162,7 +195,15 @@ public class ValidateSignatureController {
CertificateValidationService.ValidationTime validationTimeResult =
certValidationService.extractValidationTime(signerInfo);
Date validationTime;
if (validationTimeResult == null) {
if (timeStampGenTime != null) {
// The TSA's own asserted time is the authoritative one here, and is
// exactly what makes the signature verifiable after the cert expires.
validationTime = timeStampGenTime;
// Distinct from "timestamp", which CertificateValidationService already
// uses for a signature countersigned by a TSA. Both are RFC 3161, but
// one attests a signature and the other attests the whole document.
result.setValidationTimeSource("document-timestamp");
} else if (validationTimeResult == null) {
validationTime = new Date();
result.setValidationTimeSource("current");
} else {
@@ -235,10 +276,13 @@ public class ValidateSignatureController {
// Set basic signature info
result.setSignerName(sig.getName());
// A DocTimeStamp has no /M entry; its date is the TSA's genTime.
result.setSignatureDate(
sig.getSignDate() != null
? sig.getSignDate().getTime().toString()
: null);
timeStampGenTime != null
? timeStampGenTime.toString()
: sig.getSignDate() != null
? sig.getSignDate().getTime().toString()
: null);
result.setReason(sig.getReason());
result.setLocation(sig.getLocation());
@@ -301,4 +345,20 @@ public class ValidateSignatureController {
return ResponseEntity.ok(results);
}
/**
* True when the timestamp token was issued over exactly these bytes.
*
* <p>The digest algorithm is taken from the token rather than assumed, because a TSA chooses it
* - assuming SHA-256 would silently fail against any TSA that uses something else.
*/
private static boolean timestampCoversContent(TimeStampTokenInfo info, byte[] signedContent)
throws Exception {
org.bouncycastle.operator.DigestCalculator digest =
new JcaDigestCalculatorProviderBuilder().build().get(info.getHashAlgorithm());
try (java.io.OutputStream out = digest.getOutputStream()) {
out.write(signedContent);
}
return java.util.Arrays.equals(digest.getDigest(), info.getMessageImprintDigest());
}
}
app/core/src/main/java/stirling/software/SPDF/model/ApiEndpoint.java
+2 -2
View File
@@ -18,10 +18,10 @@ public class ApiEndpoint {
postNode.path("parameters")
.forEach(
paramNode -> {
String paramName = paramNode.path("name").asText("");
String paramName = paramNode.path("name").asString("");
parameters.put(paramName, paramNode);
});
this.description = postNode.path("description").asText("");
this.description = postNode.path("description").asString("");
}
public boolean areParametersValid(Map<String, Object> providedParams) {
app/core/src/main/resources/application.properties
+4 -1
View File
@@ -72,6 +72,8 @@ spring.datasource.username=sa
spring.datasource.password=
spring.h2.console.enabled=false
spring.jpa.hibernate.ddl-auto=update
# Batch associations into IN() loads so list endpoints don't N+1 as tables grow.
spring.jpa.properties.hibernate.default_batch_fetch_size=100
# Defer datasource initialization to ensure that the database is fully set up
# before Hibernate attempts to access it. This is particularly useful when
# using database initialization scripts or tools.
@@ -96,7 +98,8 @@ spring.main.allow-bean-definition-overriding=true
# spring-data-redis is on the classpath only for the optional Valkey backplane (which wires its own
# factory); exclude Spring Boot's stock Redis auto-config so a default install doesn't create a dead
# localhost:6379 factory that flips /actuator/health to DOWN.
spring.autoconfigure.exclude=org.springframework.boot.data.redis.autoconfigure.DataRedisAutoConfiguration,org.springframework.boot.data.redis.autoconfigure.DataRedisReactiveAutoConfiguration
# Also exclude the repositories auto-config: in cluster mode it needs a redisTemplate bean we don't define.
spring.autoconfigure.exclude=org.springframework.boot.data.redis.autoconfigure.DataRedisAutoConfiguration,org.springframework.boot.data.redis.autoconfigure.DataRedisReactiveAutoConfiguration,org.springframework.boot.data.redis.autoconfigure.DataRedisRepositoriesAutoConfiguration
# Set up a consistent temporary directory location
java.io.tmpdir=${stirling.tempfiles.directory:${java.io.tmpdir}/stirling-pdf}
app/core/src/main/resources/settings.yml.template
+36 -3
View File
@@ -366,6 +366,35 @@ aiEngine:
enabled: false # Set to 'true' to enable the AI engine integration
url: http://localhost:5001 # URL of the Python AI engine
timeoutSeconds: 120 # Timeout in seconds for AI engine requests
longRunningTimeoutSeconds: 600 # Timeout (seconds) for heavy operations like RAG ingestion of large documents
streamTimeoutSeconds: 1800 # SSE stream timeout (seconds) for long-running orchestrator runs
pushConfigToEngine: true # Push admin AI config to the engine on startup + save; false = engine stays fully env-controlled
models:
provider: anthropic # Model provider: 'anthropic', 'openai', 'ollama', or 'custom' (OpenAI-compatible)
smartModel: claude-haiku-4-5 # High-quality tier model name (no provider prefix)
fastModel: claude-haiku-4-5 # Cheap/fast tier model name (no provider prefix)
smartMaxTokens: 8192 # Max output tokens for the smart tier
fastMaxTokens: 2048 # Max output tokens for the fast tier
apiKey: "" # API key for the selected provider (secret). Empty = engine uses its native env credentials (e.g. ANTHROPIC_API_KEY)
baseUrl: "" # OpenAI-compatible base URL for 'ollama'/'custom' providers (e.g. http://ollama:11434/v1). Ignored for anthropic/openai
rag:
embeddingProvider: voyageai # Embedding provider: 'voyageai', 'openai', 'ollama', or 'custom' (OpenAI-compatible)
embeddingModel: voyage-4 # Embedding model name (no provider prefix)
embeddingApiKey: "" # Secret API key for the embedding provider. Empty = engine uses its native env credentials (e.g. VOYAGE_API_KEY)
embeddingBaseUrl: "" # OpenAI-compatible base URL for 'ollama'/'custom' embedding providers (e.g. http://ollama:11434/v1). Ignored for voyageai/openai
topK: 20 # Number of chunks retrieval returns per search
maxSearches: 5 # Per-run cap on knowledge-search tool calls before the agent must answer
limits:
maxPages: 200 # Upper bound on PDF pages the engine will process per request
maxCharacters: 200000 # Upper bound on characters of extracted text per request
modelMaxConcurrency: 32 # Process-wide cap on concurrent model API calls (engine restart to apply)
features: # Per-capability switches; turn an individual AI tool off without disabling the whole engine
chat: true # Assistant chat
documentQuestions: true # Ask-questions-about-a-PDF
createPdf: true # Generate a PDF from a natural-language spec
mathAuditor: true # Numerical/formula contradiction auditing
pdfComment: true # AI-authored PDF comments/annotations
classify: true # Automatic document classification/labelling
formDetection:
enabled: true # Master on/off switch for the Auto Form Detection feature
@@ -376,9 +405,11 @@ formDetection:
policies:
# Folder automations can read from and write to the directories you allow here, so treat this as a
# security boundary. Leave allowedFolderRoots empty (default) to disable folder sources/outputs
# entirely; list absolute directories to permit folder access only within them. Stirling's own
# config directory is always off-limits, and folder access is always disabled in SaaS mode.
# security boundary. Leave allowedFolderRoots empty (default) to disable folder sources/outputs,
# other than from directories that are always permitted like server file-storage and watched folders.
# List absolute directories to permit folder access within them.
# Stirling's own config directory is always off-limits, and folder access is always
# disabled in SaaS mode.
allowedFolderRoots: [] # e.g. ["/data/inbox", "/data/outbox"]
scheduleSweepSeconds: 60 # How often (seconds) scheduled policies are checked for being due
watchReconcileSeconds: 300 # How often (seconds) folder-watch re-syncs watches and re-runs as a safety net for missed events
@@ -392,6 +423,8 @@ policies:
mcp:
enabled: false # Master switch. 'false' (default) means no /mcp endpoint, no metadata, no beans wired.
scopesEnabled: true # Enforce mcp.tools.read / mcp.tools.write scopes derived from operation category
maxRequestBytes: 10485760 # Max size (bytes) of an incoming MCP tool request payload (default 10 MB)
maxInlineResponseBytes: 10485760 # Max size (bytes) of an MCP tool response returned inline before it is rejected (default 10 MB)
allowedOperations: [] # Tool allow-list (operation ids, e.g. ['compress-pdf']). Empty = all. When set, ONLY these are exposed over MCP.
blockedOperations: [] # Tool deny-list (operation ids). Always removed from MCP even if otherwise allowed.
auth:
app/core/src/test/java/stirling/software/SPDF/controller/api/security/DocumentTimestampValidationTest.java
+91
View File
@@ -0,0 +1,91 @@
package stirling.software.SPDF.controller.api.security;
import static org.assertj.core.api.Assertions.assertThat;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.Mockito.when;
import java.io.IOException;
import java.io.InputStream;
import java.util.List;
import org.apache.pdfbox.Loader;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.core.io.ClassPathResource;
import org.springframework.mock.web.MockMultipartFile;
import stirling.software.SPDF.model.api.security.SignatureValidationRequest;
import stirling.software.SPDF.model.api.security.SignatureValidationResult;
import stirling.software.SPDF.service.CertificateValidationService;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.service.CustomPDFDocumentFactory;
/**
* Validation of RFC 3161 document timestamps (PAdES-LTV).
*
* <p>These fixtures are a real PDF stamped by a real public TSA (freetsa.org). Before this was
* handled explicitly, every such timestamp was reported invalid: a DocTimeStamp's CMS encapsulates
* a TSTInfo rather than being detached over the document, so digesting the byte range compared
* against the wrong thing and always mismatched. That made the timestamp feature look broken to
* anyone who checked their own output with our validator.
*/
class DocumentTimestampValidationTest {
private ValidateSignatureController controller;
@BeforeEach
void setUp() throws Exception {
CertificateValidationService certValidationService =
new CertificateValidationService(null, new ApplicationProperties());
CustomPDFDocumentFactory factory = org.mockito.Mockito.mock(CustomPDFDocumentFactory.class);
// Delegate to the real loader so the signature dictionary is parsed as in production.
when(factory.load(any(InputStream.class)))
.thenAnswer(
invocation ->
Loader.loadPDF(
((InputStream) invocation.getArgument(0)).readAllBytes()));
controller = new ValidateSignatureController(factory, certValidationService);
}
@Test
void aGenuineDocumentTimestampValidates() throws Exception {
SignatureValidationResult result = validate("timestamp/doc-timestamped.pdf");
assertThat(result.isValid()).isTrue();
assertThat(result.getErrorMessage()).isNull();
// The TSA's asserted time is what keeps the signature verifiable once the signing
// certificate expires, so it must be the time we validate against.
// Deliberately not "timestamp" - that value already means "signature countersigned by a
// TSA", which is a different assertion about a different thing.
assertThat(result.getValidationTimeSource()).isEqualTo("document-timestamp");
assertThat(result.getSignatureDate()).isNotNull();
assertThat(result.getSubjectDN()).contains("freetsa.org");
assertThat(result.isCoversEntireDocument()).isTrue();
}
@Test
void aTamperedDocumentFailsTheMessageImprintCheck() throws Exception {
// Same file with a single byte flipped inside the signed range. Without the imprint check
// the CMS signature over the TSTInfo would still verify happily - the token is untouched -
// and a modified document would be reported as validly timestamped.
SignatureValidationResult result = validate("timestamp/doc-timestamped-tampered.pdf");
assertThat(result.isValid()).isFalse();
assertThat(result.getErrorMessage())
.isEqualTo("Timestamp message imprint does not match the document");
}
private SignatureValidationResult validate(String resource) throws IOException {
byte[] bytes;
try (InputStream in = new ClassPathResource(resource).getInputStream()) {
bytes = in.readAllBytes();
}
SignatureValidationRequest request = new SignatureValidationRequest();
request.setFileInput(
new MockMultipartFile("fileInput", "doc.pdf", "application/pdf", bytes));
List<SignatureValidationResult> results = controller.validateSignature(request).getBody();
assertThat(results).hasSize(1);
return results.get(0);
}
}
app/core/src/test/resources/timestamp/doc-timestamped-tampered.pdf
BIN
View File
Binary file not shown.
app/core/src/test/resources/timestamp/doc-timestamped.pdf
BIN
View File
Binary file not shown.
app/proprietary/build.gradle
+15 -27
View File
@@ -14,33 +14,6 @@ ext {
bootRun {
enabled = false
}
spotless {
java {
target 'src/**/java/**/*.java'
targetExclude 'src/main/java/org/apache/**'
googleJavaFormat(googleJavaFormatVersion).aosp().reorderImports(false)
// google-java-format 1.28.0 bundles Guava 32.x which crashes Spotless lint on JDK 24/25
suppressLintsFor { setStep('google-java-format') }
importOrder("java", "javax", "org", "com", "net", "io", "jakarta", "lombok", "me", "stirling")
trimTrailingWhitespace()
leadingTabsToSpaces()
endWithNewline()
}
yaml {
target '**/*.yml', '**/*.yaml'
trimTrailingWhitespace()
leadingTabsToSpaces()
endWithNewline()
}
format 'gradle', {
target '**/gradle/*.gradle', '**/*.gradle'
trimTrailingWhitespace()
leadingTabsToSpaces()
endWithNewline()
}
}
dependencies {
implementation project(':common')
api "com.google.guava:guava:${guavaVersion}"
@@ -86,6 +59,20 @@ dependencies {
runtimeOnly "com.microsoft.onnxruntime:onnxruntime:$onnxruntimeVersion"
}
// jinjava/jjwt transitively request older Jackson 2 versions; declare the current
// version directly so it is selected consistently (root build.gradle pins are the fallback).
runtimeOnly "com.fasterxml.jackson.core:jackson-core:${jackson2Version}"
runtimeOnly "com.fasterxml.jackson.core:jackson-databind:${jackson2Version}"
implementation("com.hubspot.jinjava:jinjava:${jinjavaVersion}") {
// Compile-time-only annotation artifacts (class-retention annotations, not needed at
// runtime) whose declared licences (LGPL / none) fail the licence compatibility check.
exclude group: 'com.google.code.findbugs', module: 'annotations'
exclude group: 'org.derive4j', module: 'derive4j-annotation'
exclude group: 'com.hubspot.immutables', module: 'hubspot-style'
exclude group: 'com.hubspot.immutables', module: 'immutable-collection-encodings'
}
api 'io.micrometer:micrometer-registry-prometheus'
api "io.jsonwebtoken:jjwt-api:${jwtVersion}"
@@ -109,6 +96,7 @@ dependencies {
testImplementation "org.testcontainers:testcontainers:${testcontainersMinioVersion}"
testImplementation "org.testcontainers:minio:${testcontainersMinioVersion}"
testImplementation "org.testcontainers:localstack:${testcontainersMinioVersion}"
testImplementation "org.testcontainers:postgresql:${testcontainersMinioVersion}"
testImplementation "org.testcontainers:junit-jupiter:${testcontainersMinioVersion}"
}
app/proprietary/src/main/java/stirling/software/proprietary/access/service/ResourceAccessService.java
+52 -3
View File
@@ -1,5 +1,6 @@
package stirling.software.proprietary.access.service;
import java.util.Collection;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
@@ -42,6 +43,52 @@ public class ResourceAccessService {
return canUseResource(ResourceType.PORTAL, "", null, portalDefaultPolicy, user);
}
/**
* Portal access for a roster (admin, grant, or default policy). {@code activeTeamLeaderUserIds}
* must hold ids of users who lead their own active team — the set the ADMINS_AND_TEAM_LEADS
* default admits, matching {@link #canAccessPortal}.
*/
public Set<Long> usersWithPortalAccess(
Collection<User> users, Set<Long> activeTeamLeaderUserIds) {
Set<PrincipalRef> grantedPrincipals = new HashSet<>();
for (ResourceGrant g :
grantRepository.findByResourceTypeAndResourceId(ResourceType.PORTAL, "")) {
if (permissionSatisfies(g.getPermission(), AccessPermission.USE)) {
grantedPrincipals.add(new PrincipalRef(g.getPrincipalType(), g.getPrincipalId()));
}
}
Set<Long> leaderIds = activeTeamLeaderUserIds == null ? Set.of() : activeTeamLeaderUserIds;
Set<Long> allowed = new HashSet<>();
for (User user : users) {
if (user != null
&& user.getId() != null
&& hasPortalAccess(user, grantedPrincipals, leaderIds)) {
allowed.add(user.getId());
}
}
return allowed;
}
private boolean hasPortalAccess(
User user, Set<PrincipalRef> grantedPrincipals, Set<Long> leaderIds) {
if (isAdmin(user)) {
return true;
}
for (PrincipalRef principal : principalResolver.principalsOf(user)) {
if (grantedPrincipals.contains(principal)) {
return true;
}
}
if (portalDefaultPolicy == null) {
return false;
}
return switch (portalDefaultPolicy) {
case ORG_ALL -> principalResolver.allowsDeploymentWideAccess();
case ADMINS_AND_TEAM_LEADS -> leaderIds.contains(user.getId());
case EXPLICIT_ONLY -> false;
};
}
/** Whether the user may use a resource, falling back to its default policy. */
public boolean canUseResource(
ResourceType type,
@@ -172,11 +219,13 @@ public class ResourceAccessService {
};
}
// Portal (no owner) admits any team lead; a team-owned resource admits only that team's
// leads; a user-owned resource admits no extra leads.
// Portal (no owner) admits the leader of the user's active team; a team-owned resource
// admits only that team's leads; a user-owned resource admits no extra leads.
private boolean matchesTeamLeadDefault(PrincipalRef owner, User user) {
if (owner == null) {
return teamLeadLookup.isAnyTeamLeader(user);
return user.getTeam() != null
&& user.getTeam().getId() != null
&& teamLeadLookup.isLeaderOfTeam(user, user.getTeam().getId());
}
return owner.type() == PrincipalType.TEAM
&& owner.id() != null
app/proprietary/src/main/java/stirling/software/proprietary/access/service/SecretMasker.java
+69
View File
@@ -39,6 +39,11 @@ public class SecretMasker {
"bearer",
"signature");
// Keys whose nested map holds secrets under arbitrary, caller-named keys - a free-form HTTP
// headers map is the case in point: the secret can sit under any header name (X-API-Key,
// Ocp-Apim-Subscription-Key), so the name is no signal. Mask every value in these outright.
private static final Set<String> SENSITIVE_VALUE_CONTAINERS = Set.of("headers");
/** Replace sensitive values with the mask (recursively) for safe display. */
public Map<String, Object> mask(Map<String, Object> config) {
return mask(config, 0);
@@ -73,6 +78,12 @@ public class SecretMasker {
if (isSensitive(e.getKey()) && isRedacted(e.getValue(), depth)) {
continue;
}
if (isSensitiveContainer(e.getKey())
&& e.getValue() instanceof Map<?, ?> m
&& depth < MAX_DEPTH) {
out.put(e.getKey(), sanitizeAllValues(castMap(m), depth + 1));
continue;
}
out.put(
e.getKey(),
e.getValue() instanceof Map<?, ?> m && depth < MAX_DEPTH
@@ -100,6 +111,14 @@ public class SecretMasker {
}
continue;
}
if (isSensitiveContainer(key)
&& depth < MAX_DEPTH
&& stored.get(key) instanceof Map<?, ?> s
&& value instanceof Map<?, ?> i) {
// Every value here is a secret, so restore a redacted one from stored per-entry.
out.put(key, mergeAllValues(castMap(s), castMap(i), depth + 1));
continue;
}
if (depth < MAX_DEPTH
&& stored.get(key) instanceof Map<?, ?> s
&& value instanceof Map<?, ?> i) {
@@ -119,6 +138,9 @@ public class SecretMasker {
}
return MASK;
}
if (isSensitiveContainer(key) && value instanceof Map<?, ?> m && depth < MAX_DEPTH) {
return maskAllValues(castMap(m), depth + 1);
}
if (depth >= MAX_DEPTH) {
// Too deep to descend; mask containers rather than risk leaking an unmasked secret.
return value instanceof Map<?, ?> || value instanceof List<?> ? MASK : value;
@@ -141,6 +163,53 @@ public class SecretMasker {
return SENSITIVE_HINTS.stream().anyMatch(lower::contains);
}
private boolean isSensitiveContainer(String key) {
return SENSITIVE_VALUE_CONTAINERS.contains(key.toLowerCase(Locale.ROOT));
}
/** Mask every value in a container map, whatever its keys are named. */
private Map<String, Object> maskAllValues(Map<String, Object> map, int depth) {
Map<String, Object> out = new LinkedHashMap<>();
for (Map.Entry<String, Object> e : map.entrySet()) {
Object v = e.getValue();
if (v == null || (v instanceof String s && s.isBlank())) {
out.put(e.getKey(), v);
} else if (v instanceof Map<?, ?> m && depth < MAX_DEPTH) {
out.put(e.getKey(), maskAllValues(castMap(m), depth + 1));
} else {
out.put(e.getKey(), MASK);
}
}
return out;
}
/** Merge a container map treating every entry as a secret, restoring redacted from stored. */
private Map<String, Object> mergeAllValues(
Map<String, Object> stored, Map<String, Object> incoming, int depth) {
Map<String, Object> out = new LinkedHashMap<>();
for (Map.Entry<String, Object> e : incoming.entrySet()) {
if (isRedacted(e.getValue(), depth)) {
if (stored.containsKey(e.getKey())) {
out.put(e.getKey(), stored.get(e.getKey()));
}
} else {
out.put(e.getKey(), e.getValue());
}
}
return out;
}
/** Drop redacted entries from a container map on create, whatever their keys are named. */
private Map<String, Object> sanitizeAllValues(Map<String, Object> map, int depth) {
Map<String, Object> out = new LinkedHashMap<>();
for (Map.Entry<String, Object> e : map.entrySet()) {
if (!isRedacted(e.getValue(), depth)) {
out.put(e.getKey(), e.getValue());
}
}
return out;
}
/** Blank, the mask placeholder, or any structure that still contains the mask. */
private boolean isRedacted(Object value, int depth) {
if (value == null) {
app/proprietary/src/main/java/stirling/software/proprietary/accountlink/InstanceEntitlementInterceptor.java
+8 -1
View File
@@ -35,6 +35,7 @@ import stirling.software.proprietary.billing.ContentHasher;
import stirling.software.proprietary.billing.DocumentUnitCalculator;
import stirling.software.proprietary.billing.DocumentUnitCalculator.FileSize;
import stirling.software.proprietary.billing.UnitCalcPolicy;
import stirling.software.proprietary.policy.controller.PolicyRunRoutes;
import stirling.software.proprietary.security.model.ApiKeyAuthenticationToken;
/**
@@ -84,7 +85,13 @@ public class InstanceEntitlementInterceptor implements HandlerInterceptor {
instanceof ApiKeyAuthenticationToken;
BillingCategory category = BillableOperationClassifier.categorize(request, apiKey);
request.setAttribute(ATTR_CATEGORY, category);
decision = gate.evaluate(category != BillingCategory.BYPASSED);
// A policy run kicks off billable automation, so block it up front when unentitled
// rather than after its first tool. It carries no automation header itself (category
// BYPASSED), so it's gated here but metered only via its dispatched sub-steps - keeping
// the BYPASSED meter category avoids double-counting.
boolean billable =
category != BillingCategory.BYPASSED || PolicyRunRoutes.matches(request);
decision = gate.evaluate(billable);
} catch (RuntimeException e) {
// Fail open: an inability to resolve entitlement (e.g. a DB or SaaS blip) must never
// turn into a hard block on billable work.
app/proprietary/src/main/java/stirling/software/proprietary/classification/ClassificationLabelProvider.java
+61
View File
@@ -0,0 +1,61 @@
package stirling.software.proprietary.classification;
import java.io.InputStream;
import java.util.List;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.core.io.ClassPathResource;
import org.springframework.stereotype.Component;
import lombok.extern.slf4j.Slf4j;
import stirling.software.proprietary.classification.model.ClassificationLabel;
import stirling.software.proprietary.classification.model.ClassificationLabels;
import tools.jackson.databind.ObjectMapper;
/**
* Supplies the classification vocabulary the classify tool sends to the AI engine. The set is a
* fixed, built-in list bundled with the application ({@code
* classification/classification-labels.json}) and shared by everyone — there is no per-team
* customization or database. Loaded once at startup.
*/
@Slf4j
@Component
public class ClassificationLabelProvider {
private static final String RESOURCE = "classification/classification-labels.json";
private final List<ClassificationLabel> labels;
// Explicit @Autowired: the class has a second (private) constructor for tests, so Spring
// can't infer which to use without it.
@Autowired
public ClassificationLabelProvider(ObjectMapper objectMapper) {
this(load(objectMapper));
}
private ClassificationLabelProvider(List<ClassificationLabel> labels) {
this.labels = List.copyOf(labels);
}
/** Build a provider with an explicit label set (tests). */
public static ClassificationLabelProvider withLabels(List<ClassificationLabel> labels) {
return new ClassificationLabelProvider(labels);
}
/** The built-in vocabulary, in file order. */
public List<ClassificationLabel> labels() {
return labels;
}
private static List<ClassificationLabel> load(ObjectMapper objectMapper) {
try (InputStream in = new ClassPathResource(RESOURCE).getInputStream()) {
ClassificationLabels parsed = objectMapper.readValue(in, ClassificationLabels.class);
return parsed.labels();
} catch (Exception e) {
log.error("Failed to load classification labels from {}", RESOURCE, e);
return List.of();
}
}
}
app/proprietary/src/main/java/stirling/software/proprietary/classification/ClassificationLabelsController.java
-136
View File
@@ -1,136 +0,0 @@
package stirling.software.proprietary.classification;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.DeleteMapping;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PutMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.server.ResponseStatusException;
import io.swagger.v3.oas.annotations.Hidden;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.tags.Tag;
import lombok.RequiredArgsConstructor;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.service.UserServiceInterface;
import stirling.software.proprietary.classification.model.ClassificationLabels;
import stirling.software.proprietary.classification.model.LabelsValidator;
import stirling.software.proprietary.classification.store.ClassificationLabelStore;
import stirling.software.proprietary.classification.store.TeamLabelsEntity;
import stirling.software.proprietary.policy.config.PolicyManagementAuthority;
/**
* Read/write the team's classification label set — the flat vocabulary the document classifier runs
* against. Shared and team-scoped exactly like policies: every user reads their own team's labels,
* and only a user who may edit policies (a team leader on SaaS, the global admin self-hosted; see
* {@link PolicyManagementAuthority}) may change it — gated only when login is enabled, since
* single-user deployments trust the local operator. A team with no stored labels reads as {@code
* 204}; that team has no vocabulary, so its documents are not classified (there is no built-in
* default on the backend or the engine — the label data lives only in the frontend).
*/
@RestController
@RequestMapping("/api/v1/classification/labels")
@Hidden
@RequiredArgsConstructor
@Tag(name = "Classification", description = "Team-scoped document-classification labels")
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class ClassificationLabelsController {
private final ClassificationLabelStore labelStore;
private final PolicyManagementAuthority policyManagementAuthority;
private final ApplicationProperties applicationProperties;
private final UserServiceInterface userService;
@GetMapping
@Operation(
summary = "Get the team's classification labels",
description =
"Returns the caller's team label set, or 204 when the team has none (its"
+ " documents are then not classified).")
public ResponseEntity<ClassificationLabels> getTeamLabels() {
return labelStore
.findByTeam(currentTeamId())
.map(ResponseEntity::ok)
.orElseGet(() -> ResponseEntity.noContent().build());
}
@PutMapping(consumes = MediaType.APPLICATION_JSON_VALUE)
@Operation(
summary = "Save the team's classification labels",
description =
"Validates and stores the label set for the caller's team, shared by everyone"
+ " on the team. Requires the policy-editor role for the team.")
public ResponseEntity<ClassificationLabels> saveTeamLabels(
@RequestBody ClassificationLabels labels) {
requireEditingAllowed();
validate(labels);
ClassificationLabels saved = labelStore.save(currentTeamId(), labels, currentUsername());
return ResponseEntity.ok(saved);
}
@DeleteMapping
@Operation(
summary = "Reset the team's classification labels",
description =
"Removes the team's stored label set; its documents are then not classified"
+ " until labels are saved again. Requires the policy-editor role for the"
+ " team.")
public ResponseEntity<Void> resetTeamLabels() {
requireEditingAllowed();
labelStore.deleteByTeam(currentTeamId());
return ResponseEntity.noContent().build();
}
private static void validate(ClassificationLabels labels) {
try {
LabelsValidator.validate(labels);
} catch (IllegalArgumentException e) {
throw new ResponseStatusException(HttpStatus.BAD_REQUEST, e.getMessage());
}
}
/**
* Editing the team labels requires the editor role for the caller's team — the same gate
* policies use (team leader on SaaS, global admin self-hosted). Single-user deployments (login
* disabled) have no such role, so they trust the local operator.
*/
private void requireEditingAllowed() {
if (!applicationProperties.getSecurity().isEnableLogin()) {
return;
}
if (!policyManagementAuthority.canEditPolicies()) {
throw new ResponseStatusException(
HttpStatus.FORBIDDEN,
"The team classification labels may only be changed by a team leader");
}
}
/**
* The caller's team key. With login disabled the single operator owns the {@link
* TeamLabelsEntity#NO_TEAM} sentinel row; with login enabled a caller with no resolvable team
* is an error rather than being dropped into the shared sentinel bucket (which would let
* unteamed users read and overwrite each other's "team" labels).
*/
private Long currentTeamId() {
Long teamId = policyManagementAuthority.currentUserTeamId();
if (teamId != null) {
return teamId;
}
if (!applicationProperties.getSecurity().isEnableLogin()) {
return TeamLabelsEntity.NO_TEAM;
}
throw new ResponseStatusException(
HttpStatus.UNAUTHORIZED, "Could not resolve the current user's team");
}
private String currentUsername() {
return userService == null ? null : userService.getCurrentUsername();
}
}
app/proprietary/src/main/java/stirling/software/proprietary/classification/ClassificationRunBiller.java
+8
View File
@@ -0,0 +1,8 @@
package stirling.software.proprietary.classification;
/** Meters a client-side classification run; SaaS charges PAYG, other flavors have no bean. */
public interface ClassificationRunBiller {
/** Charge one classification policy run covering {@code documentCount} documents. */
void recordClassificationRun(int documentCount);
}
app/proprietary/src/main/java/stirling/software/proprietary/classification/model/ClassificationLabels.java
+4 -4
View File
@@ -3,10 +3,10 @@ package stirling.software.proprietary.classification.model;
import java.util.List;
/**
* A flat multi-label classification vocabulary — the set of labels a document may be assigned.
* Stored per team (admin-edited, shared by everyone on the team); the classifier runs against these
* label names. A team with no stored set has no vocabulary, so its documents are not classified —
* neither the backend nor the engine holds a default of its own.
* A flat multi-label classification vocabulary — the set of labels a document may be assigned. The
* classifier runs against these label names. The vocabulary is a fixed, built-in set shared by
* everyone (see {@link stirling.software.proprietary.classification.ClassificationLabelProvider});
* this record is the JSON parse target for that bundled resource.
*/
public record ClassificationLabels(List<ClassificationLabel> labels) {
app/proprietary/src/main/java/stirling/software/proprietary/classification/model/LabelsValidator.java
-71
View File
@@ -1,71 +0,0 @@
package stirling.software.proprietary.classification.model;
import java.util.HashSet;
import java.util.Locale;
import java.util.Set;
import java.util.regex.Pattern;
/**
* Structural validation for a user- or admin-supplied label set, run before it is stored so a
* malformed vocabulary can never reach the classifier. Mirrors the invariants the engine relies on:
* non-blank ids and names, each unique within the set (ids exactly, names case-insensitively).
*/
public final class LabelsValidator {
private LabelsValidator() {}
// Generous upper bounds so a legitimate label set is never blocked, but a single team or user
// can't store an unbounded blob that would bloat the row, balloon the classifier prompt, or
// exhaust memory on deserialize.
static final int MAX_LABELS = 500;
static final int MAX_TEXT_LENGTH = 128;
// Icon is a Material Symbols key (lowercase, digits, hyphens). Enforce the SHAPE server-side —
// the exact allowlist lives in the frontend — so a client bypassing the UI can't store
// arbitrary
// text that would render as garbage (or worse) in every teammate's sidebar.
private static final Pattern ICON_KEY = Pattern.compile("^[a-z0-9-]+$");
/**
* @throws IllegalArgumentException with a human-readable message when the label set is invalid.
*/
public static void validate(ClassificationLabels labels) {
if (labels == null || labels.labels() == null) {
throw new IllegalArgumentException("Labels are required");
}
if (labels.labels().size() > MAX_LABELS) {
throw new IllegalArgumentException("Too many labels (max " + MAX_LABELS + ")");
}
Set<String> ids = new HashSet<>();
Set<String> names = new HashSet<>();
for (ClassificationLabel label : labels.labels()) {
requireText(label.id(), "Label id");
requireText(label.name(), "Label name");
if (label.icon() != null && !label.icon().isEmpty()) {
if (label.icon().length() > MAX_TEXT_LENGTH) {
throw new IllegalArgumentException(
"Label icon is too long (max " + MAX_TEXT_LENGTH + " characters)");
}
if (!ICON_KEY.matcher(label.icon()).matches()) {
throw new IllegalArgumentException("Invalid label icon: " + label.icon());
}
}
if (!ids.add(label.id().trim())) {
throw new IllegalArgumentException("Duplicate label id: " + label.id());
}
if (!names.add(label.name().trim().toLowerCase(Locale.ROOT))) {
throw new IllegalArgumentException("Duplicate label name: " + label.name());
}
}
}
private static void requireText(String value, String field) {
if (value == null || value.isBlank()) {
throw new IllegalArgumentException(field + " must not be blank");
}
if (value.trim().length() > MAX_TEXT_LENGTH) {
throw new IllegalArgumentException(
field + " is too long (max " + MAX_TEXT_LENGTH + " characters)");
}
}
}
app/proprietary/src/main/java/stirling/software/proprietary/classification/store/ClassificationLabelStore.java
-22
View File
@@ -1,22 +0,0 @@
package stirling.software.proprietary.classification.store;
import java.util.Optional;
import stirling.software.proprietary.classification.model.ClassificationLabels;
/**
* Stores one {@link ClassificationLabels} set per team. A {@code null} teamId addresses the
* unteamed set (login disabled / no resolvable team), mirroring how the policy store treats a null
* team.
*/
public interface ClassificationLabelStore {
/** The team's stored labels, or empty when it has none (callers then skip classification). */
Optional<ClassificationLabels> findByTeam(Long teamId);
/** Create or replace the team's labels. Returns the stored value. */
ClassificationLabels save(Long teamId, ClassificationLabels labels, String updatedBy);
/** Remove the team's labels (reset to default). Returns whether a set existed. */
boolean deleteByTeam(Long teamId);
}
app/proprietary/src/main/java/stirling/software/proprietary/classification/store/InProcessClassificationLabelStore.java
-36
View File
@@ -1,36 +0,0 @@
package stirling.software.proprietary.classification.store;
import java.util.Map;
import java.util.Optional;
import java.util.concurrent.ConcurrentHashMap;
import stirling.software.proprietary.classification.model.ClassificationLabels;
/**
* In-memory {@link ClassificationLabelStore} for tests and any future no-database mode. {@link
* JpaClassificationLabelStore} is the runtime bean.
*/
public class InProcessClassificationLabelStore implements ClassificationLabelStore {
private final Map<Long, ClassificationLabels> byTeam = new ConcurrentHashMap<>();
@Override
public Optional<ClassificationLabels> findByTeam(Long teamId) {
return Optional.ofNullable(byTeam.get(key(teamId)));
}
@Override
public ClassificationLabels save(Long teamId, ClassificationLabels labels, String updatedBy) {
byTeam.put(key(teamId), labels);
return labels;
}
@Override
public boolean deleteByTeam(Long teamId) {
return byTeam.remove(key(teamId)) != null;
}
private static long key(Long teamId) {
return teamId == null ? TeamLabelsEntity.NO_TEAM : teamId;
}
}
app/proprietary/src/main/java/stirling/software/proprietary/classification/store/JpaClassificationLabelStore.java
-76
View File
@@ -1,76 +0,0 @@
package stirling.software.proprietary.classification.store;
import java.time.Instant;
import java.util.Optional;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.stereotype.Service;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.proprietary.classification.model.ClassificationLabels;
import tools.jackson.core.JacksonException;
import tools.jackson.databind.ObjectMapper;
/**
* Durable {@link ClassificationLabelStore} backed by JPA; the runtime store. Gated on {@code
* policies.enabled} — stored labels only matter when the Classification policy can run — so it
* shares the policy subsystem's on/off switch. Each label set is persisted as JSON via {@link
* TeamLabelsEntity}.
*/
@Slf4j
@Service
@RequiredArgsConstructor
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class JpaClassificationLabelStore implements ClassificationLabelStore {
private final TeamLabelsRepository teamRepository;
private final ObjectMapper objectMapper;
@Override
public Optional<ClassificationLabels> findByTeam(Long teamId) {
return teamRepository
.findById(key(teamId))
.flatMap(entity -> parse(entity.getLabelsJson(), "team " + teamId));
}
@Override
public ClassificationLabels save(Long teamId, ClassificationLabels labels, String updatedBy) {
TeamLabelsEntity entity = new TeamLabelsEntity();
entity.setTeamId(key(teamId));
entity.setLabelsJson(objectMapper.writeValueAsString(labels));
entity.setUpdatedAt(Instant.now());
entity.setUpdatedBy(updatedBy);
teamRepository.save(entity);
return labels;
}
@Override
public boolean deleteByTeam(Long teamId) {
long id = key(teamId);
if (!teamRepository.existsById(id)) {
return false;
}
teamRepository.deleteById(id);
return true;
}
private Optional<ClassificationLabels> parse(String json, String owner) {
try {
return Optional.of(objectMapper.readValue(json, ClassificationLabels.class));
} catch (JacksonException e) {
// A stored label set that no longer parses (corruption / manual DB edit) must not break
// classification: drop it so the caller treats the team as having no labels (and skips
// classification) rather than surfacing a 500 on every upload.
log.warn("Discarding unparseable stored labels for {}: {}", owner, e.getMessage());
return Optional.empty();
}
}
/** Map the nullable team id onto the entity's non-null key (sentinel for the unteamed case). */
private static long key(Long teamId) {
return teamId == null ? TeamLabelsEntity.NO_TEAM : teamId;
}
}
app/proprietary/src/main/java/stirling/software/proprietary/classification/store/TeamLabelsEntity.java
-47
View File
@@ -1,47 +0,0 @@
package stirling.software.proprietary.classification.store;
import java.io.Serializable;
import java.time.Instant;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.Id;
import jakarta.persistence.Table;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
/**
* JPA row for a team's classification labels — one row per team. The label set lives as JSON in
* {@code labelsJson} (authoritative on read). {@code teamId} is the natural key; the sentinel
* {@link #NO_TEAM} stands in for the unteamed (login-disabled / self-hosted single-team) case,
* since a primary key can't be null (policies store a nullable {@code team_id}, but this table is
* keyed one-per-team). Kept decoupled from the security entities — {@code teamId} is a plain value,
* not a foreign key — so classification can be enabled or disabled without touching them.
*/
@Entity
@Table(name = "classification_labels")
@NoArgsConstructor
@Getter
@Setter
public class TeamLabelsEntity implements Serializable {
private static final long serialVersionUID = 1L;
/** Sentinel key for the unteamed label set (login disabled / no resolvable team). */
public static final long NO_TEAM = 0L;
@Id
@Column(name = "team_id")
private long teamId;
@Column(name = "labels_json", columnDefinition = "text")
private String labelsJson;
@Column(name = "updated_at")
private Instant updatedAt;
@Column(name = "updated_by")
private String updatedBy;
}
app/proprietary/src/main/java/stirling/software/proprietary/classification/store/TeamLabelsRepository.java
-7
View File
@@ -1,7 +0,0 @@
package stirling.software.proprietary.classification.store;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.stereotype.Repository;
@Repository
public interface TeamLabelsRepository extends JpaRepository<TeamLabelsEntity, Long> {}
app/proprietary/src/main/java/stirling/software/proprietary/config/CustomAuditEventRepository.java
+13 -2
View File
@@ -54,10 +54,21 @@ public class CustomAuditEventRepository implements AuditEventRepository {
return;
}
String rid = MDC.get("requestId");
String apiKeyLabel =
MDC.get(
stirling.software.proprietary.security.service
.ApiKeyAuthenticationService.AUDIT_LABEL_MDC_KEY);
if (rid != null) {
if (rid != null || apiKeyLabel != null) {
clean = new java.util.HashMap<>(clean);
clean.put("requestId", rid);
if (rid != null) {
clean.put("requestId", rid);
}
// Named key that made the request; surfaces as the doc source in the processor
// feed.
if (apiKeyLabel != null) {
clean.put("__apiKeyLabel", apiKeyLabel);
}
}
String source = MDC.get("auditSource");
app/proprietary/src/main/java/stirling/software/proprietary/controller/api/AiEngineController.java
+15 -6
View File
@@ -7,7 +7,6 @@ import java.util.concurrent.Executor;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
@@ -28,6 +27,7 @@ import jakarta.validation.Valid;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.model.job.ResultFile;
import stirling.software.common.service.JobOwnershipService;
import stirling.software.common.service.TaskManager;
@@ -38,6 +38,7 @@ import stirling.software.proprietary.model.api.ai.AiWorkflowResponse;
import stirling.software.proprietary.model.api.ai.AiWorkflowResultFile;
import stirling.software.proprietary.service.AiEngineClient;
import stirling.software.proprietary.service.AiEngineEndpointResolver;
import stirling.software.proprietary.service.AiFeatureGate;
import stirling.software.proprietary.service.AiWorkflowService;
import tools.jackson.core.JacksonException;
@@ -60,15 +61,14 @@ public class AiEngineController {
private final TaskManager taskManager;
private final JobOwnershipService jobOwnershipService;
private final AiEngineEndpointResolver endpointResolver;
private final AiFeatureGate aiFeatureGate;
private final UserServiceInterface userService;
/**
* SSE emitter timeout. Long enough to accommodate multi-gigabyte PDF workflows (OCR on a
* 1000-page scan, splitting a huge PDF, etc.) without the emitter completing out from under the
* executor. Configurable via {@code stirling.ai.streamTimeoutMs}.
* SSE emitter timeout (ms), long enough for multi-gigabyte PDF workflows without completing out
* from under the executor. Derived from {@code aiEngine.streamTimeoutSeconds}.
*/
@Value("${stirling.ai.streamTimeoutMs:1800000}")
private long streamTimeoutMs;
private final long streamTimeoutMs;
public AiEngineController(
AiEngineClient aiEngineClient,
@@ -78,6 +78,8 @@ public class AiEngineController {
TaskManager taskManager,
JobOwnershipService jobOwnershipService,
AiEngineEndpointResolver endpointResolver,
AiFeatureGate aiFeatureGate,
ApplicationProperties applicationProperties,
@Autowired(required = false) UserServiceInterface userService) {
this.aiEngineClient = aiEngineClient;
this.aiWorkflowService = aiWorkflowService;
@@ -86,7 +88,10 @@ public class AiEngineController {
this.taskManager = taskManager;
this.jobOwnershipService = jobOwnershipService;
this.endpointResolver = endpointResolver;
this.aiFeatureGate = aiFeatureGate;
this.userService = userService;
this.streamTimeoutMs =
applicationProperties.getAiEngine().getStreamTimeoutSeconds() * 1000L;
}
private String currentUserId() {
@@ -111,6 +116,7 @@ public class AiEngineController {
+ " system and downloadable via GET /api/v1/general/files/{fileId}.")
public AiWorkflowResponse orchestrate(@Valid @ModelAttribute AiWorkflowRequest request)
throws IOException {
aiFeatureGate.requireConversationalWorkflow();
AiWorkflowResponse result = aiWorkflowService.orchestrate(request);
registerFileResultAsJob(result);
return result;
@@ -123,6 +129,7 @@ public class AiEngineController {
"Accepts a PDF upload and a user message, returns SSE events with progress"
+ " updates followed by the final AI workflow result")
public SseEmitter orchestrateStream(@Valid @ModelAttribute AiWorkflowRequest request) {
aiFeatureGate.requireConversationalWorkflow();
SseEmitter emitter = new SseEmitter(streamTimeoutMs);
emitter.onTimeout(
@@ -246,6 +253,8 @@ public class AiEngineController {
"Sends a user message to the PDF edit agent which returns a structured plan"
+ " of tool operations to perform")
public ResponseEntity<String> pdfEdit(@RequestBody String requestBody) throws IOException {
// Same gate as /orchestrate: edit agent is a model call on the same conversational surface.
aiFeatureGate.requireConversationalWorkflow();
JsonNode parsed = parseJson(requestBody);
if (!parsed.isObject()) {
throw new ResponseStatusException(
app/proprietary/src/main/java/stirling/software/proprietary/controller/api/ClassifyLabelController.java
+19 -32
View File
@@ -31,11 +31,11 @@ import stirling.software.common.service.PdfMetadataService;
import stirling.software.common.service.UserServiceInterface;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
import stirling.software.proprietary.classification.ClassificationLabelProvider;
import stirling.software.proprietary.classification.model.ClassificationLabel;
import stirling.software.proprietary.classification.store.ClassificationLabelStore;
import stirling.software.proprietary.model.api.ai.AiPageText;
import stirling.software.proprietary.policy.config.PolicyManagementAuthority;
import stirling.software.proprietary.service.AiEngineClient;
import stirling.software.proprietary.service.AiFeatureGate;
import stirling.software.proprietary.service.PdfContentExtractor;
import tools.jackson.databind.JsonNode;
@@ -46,7 +46,7 @@ import tools.jackson.databind.node.ObjectNode;
* Dispatchable tool that classifies a PDF and writes the result into its metadata.
*
* <p>Runs as a Classification-policy pipeline step: it reads a bounded page window, asks the AI
* engine to classify the document against the caller's team label set, and stores the engine's JSON
* engine to classify the document against the built-in label set, and stores the engine's JSON
* answer — minus the transport-only {@code outcome} field — in the custom Info-dictionary key
* {@link PdfMetadataService#CLASSIFICATION_KEY}. Returns the labelled PDF. Not intended for direct
* client use.
@@ -68,17 +68,14 @@ public class ClassifyLabelController {
private final PdfContentExtractor pdfContentExtractor;
private final PdfMetadataService pdfMetadataService;
private final AiEngineClient aiEngineClient;
private final AiFeatureGate aiFeatureGate;
private final ObjectMapper objectMapper;
private final UserServiceInterface userService;
/**
* Present only when the policy subsystem is enabled ({@code policies.enabled}); the store and
* team authority are gated on it. Null otherwise, in which case there are no team labels to
* classify against and the document is passed through unlabelled.
* The fixed, built-in vocabulary shared by everyone — see {@link ClassificationLabelProvider}.
*/
private final ClassificationLabelStore labelStore;
private final PolicyManagementAuthority policyManagementAuthority;
private final ClassificationLabelProvider labelProvider;
public ClassifyLabelController(
CustomPDFDocumentFactory pdfDocumentFactory,
@@ -86,19 +83,19 @@ public class ClassifyLabelController {
PdfContentExtractor pdfContentExtractor,
PdfMetadataService pdfMetadataService,
AiEngineClient aiEngineClient,
AiFeatureGate aiFeatureGate,
ObjectMapper objectMapper,
@Autowired(required = false) UserServiceInterface userService,
@Autowired(required = false) ClassificationLabelStore labelStore,
@Autowired(required = false) PolicyManagementAuthority policyManagementAuthority) {
ClassificationLabelProvider labelProvider,
@Autowired(required = false) UserServiceInterface userService) {
this.pdfDocumentFactory = pdfDocumentFactory;
this.tempFileManager = tempFileManager;
this.pdfContentExtractor = pdfContentExtractor;
this.pdfMetadataService = pdfMetadataService;
this.aiEngineClient = aiEngineClient;
this.aiFeatureGate = aiFeatureGate;
this.objectMapper = objectMapper;
this.labelProvider = labelProvider;
this.userService = userService;
this.labelStore = labelStore;
this.policyManagementAuthority = policyManagementAuthority;
}
@PostMapping(value = "/classify-and-label", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
@@ -111,14 +108,15 @@ public class ClassifyLabelController {
+ " intended for direct client use.")
public ResponseEntity<Resource> classifyAndLabel(
@RequestParam("fileInput") MultipartFile fileInput) throws IOException {
aiFeatureGate.requireClassify();
try (PDDocument document = pdfDocumentFactory.load(fileInput, true)) {
String fileName = safeFileName(fileInput.getOriginalFilename());
List<EngineLabel> allowed = resolveAllowedLabels();
if (allowed.isEmpty()) {
// No vocabulary to classify against (the team stored no labels): pass the file
// through unlabelled rather than ask the engine to classify against nothing.
log.debug("[classify-and-label] {} has no team labels; skipping", fileName);
// No vocabulary to classify against: pass the file through unlabelled rather than
// ask the engine to classify against nothing.
log.debug("[classify-and-label] {} has no labels; skipping", fileName);
return WebResponseUtils.pdfDocToWebResponse(document, fileName, tempFileManager);
}
@@ -175,24 +173,13 @@ public class ClassifyLabelController {
}
/**
* The allowed labels for the caller's team as {@code {id, name}} pairs, de-duplicated by id.
* The engine shows the model the names and returns the ids (icons are presentational and never
* sent). Returns an empty list — the caller then skips classification — when the policy
* subsystem is disabled (no store) or the team has no stored labels. The engine holds no
* default vocabulary of its own, so a team's stored labels are the only source.
* The built-in vocabulary as {@code {id, name}} pairs, de-duplicated by id. The engine shows
* the model the names and returns the ids (icons are presentational and never sent). The engine
* holds no default vocabulary of its own, so this bundled set is the only source.
*/
private List<EngineLabel> resolveAllowedLabels() {
if (labelStore == null) {
return List.of();
}
Long teamId =
policyManagementAuthority == null
? null
: policyManagementAuthority.currentUserTeamId();
Map<String, EngineLabel> byId = new LinkedHashMap<>();
labelStore.findByTeam(teamId).ifPresent(labels -> collectLabels(labels.labels(), byId));
collectLabels(labelProvider.labels(), byId);
return List.copyOf(byId.values());
}
app/proprietary/src/main/java/stirling/software/proprietary/controller/api/CreatePdfAgentController.java
+40 -17
View File
@@ -8,12 +8,14 @@ import java.util.List;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.springframework.core.io.Resource;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.server.ResponseStatusException;
import io.github.pixee.security.Filenames;
import io.swagger.v3.oas.annotations.Hidden;
@@ -24,18 +26,25 @@ import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.configuration.RuntimePathConfig;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.ProcessExecutor;
import stirling.software.common.util.TempFile;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
import stirling.software.proprietary.model.api.ai.create.AiDocument;
import stirling.software.proprietary.service.AiDocumentHtmlRenderer;
import stirling.software.proprietary.service.AiFeatureGate;
import tools.jackson.core.JacksonException;
import tools.jackson.databind.ObjectMapper;
/**
* Dispatchable tool that converts an AI-generated HTML string to a PDF via WeasyPrint.
* Dispatchable tool that converts an AI-generated document model to a PDF via WeasyPrint.
*
* <p>Called by {@link stirling.software.proprietary.service.AiWorkflowService} when the engine
* emits a {@code CREATE_PDF_FROM_HTML_AGENT} plan step. The HTML comes from a trusted Jinja
* template so sanitization is intentionally skipped.
* emits a {@code CREATE_PDF_FROM_HTML_AGENT} plan step. The engine supplies the document as
* structured fields; the HTML is built here from a fixed template.
*/
@Slf4j
@Hidden
@@ -48,6 +57,10 @@ public class CreatePdfAgentController {
private final TempFileManager tempFileManager;
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final RuntimePathConfig runtimePathConfig;
private final ApplicationProperties applicationProperties;
private final ObjectMapper objectMapper;
private final AiDocumentHtmlRenderer htmlRenderer;
private final AiFeatureGate aiFeatureGate;
/**
* Returns true only when WeasyPrint is definitively unavailable — either the binary could not
@@ -74,32 +87,42 @@ public class CreatePdfAgentController {
value = "/create-pdf-from-html-agent",
consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
@Operation(
summary = "Convert AI-generated HTML to a PDF",
summary = "Convert an AI-generated document to a PDF",
description =
"Accepts an HTML document as a plain-text parameter and returns a PDF."
+ " This endpoint is dispatched by the AI workflow orchestrator as a"
+ " plan step; it is not intended for direct client use.")
public ResponseEntity<Resource> createPdfFromHtml(
@RequestParam("htmlContent") String htmlContent,
@RequestParam("filename") String filename)
"Accepts a structured document as a JSON parameter and returns a PDF. This"
+ " endpoint is dispatched by the AI workflow orchestrator as a plan"
+ " step; it is not intended for direct client use.")
public ResponseEntity<Resource> createPdf(
@RequestParam("document") String document, @RequestParam("filename") String filename)
throws Exception {
if (!applicationProperties.getAiEngine().isEnabled()) {
throw new ResponseStatusException(HttpStatus.NOT_FOUND);
}
aiFeatureGate.requireCreatePdf();
AiDocument model;
try {
model = objectMapper.readValue(document, AiDocument.class);
} catch (JacksonException e) {
throw new ResponseStatusException(HttpStatus.BAD_REQUEST);
}
String html = htmlRenderer.render(model);
log.info(
"[create-pdf-agent] converting HTML to PDF via WeasyPrint — html_bytes={}",
htmlContent.length());
"[create-pdf-agent] converting document to PDF via WeasyPrint — html_bytes={}",
html.length());
try (TempFile htmlFile = tempFileManager.createManagedTempFile(".html");
TempFile pdfFile = tempFileManager.createManagedTempFile(".pdf")) {
Files.writeString(htmlFile.getPath(), htmlContent, StandardCharsets.UTF_8);
Files.writeString(htmlFile.getPath(), html, StandardCharsets.UTF_8);
List<String> command = new ArrayList<>();
command.add(runtimePathConfig.getWeasyPrintPath());
command.add("-e");
command.add("utf-8");
command.add("-v");
// SSRF: the HTML is self-contained and the engine validates style colours, so no
// external url() reaches WeasyPrint. For full isolation, run it network-isolated.
command.add(htmlFile.getAbsolutePath());
command.add(pdfFile.getAbsolutePath());
@@ -126,8 +149,8 @@ public class CreatePdfAgentController {
// avoids materialising the whole document as a byte[] twice (read-all + re-serialise),
// which matters for large generated documents.
TempFile tempOut = tempFileManager.createManagedTempFile(".pdf");
try (PDDocument document = pdfDocumentFactory.load(pdfFile.getPath())) {
document.save(tempOut.getPath().toFile());
try (PDDocument pdDocument = pdfDocumentFactory.load(pdfFile.getPath())) {
pdDocument.save(tempOut.getPath().toFile());
} catch (Exception e) {
tempOut.close();
throw e;
app/proprietary/src/main/java/stirling/software/proprietary/controller/api/MathAuditorAgentController.java
+8 -2
View File
@@ -2,6 +2,7 @@ package stirling.software.proprietary.controller.api;
import java.io.IOException;
import java.math.BigDecimal;
import java.util.regex.Pattern;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
@@ -20,6 +21,7 @@ import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.proprietary.model.api.ai.Verdict;
import stirling.software.proprietary.service.AiFeatureGate;
import stirling.software.proprietary.service.AiToolInputValidator;
import stirling.software.proprietary.service.MathAuditorOrchestrator;
@@ -46,7 +48,9 @@ import stirling.software.proprietary.service.MathAuditorOrchestrator;
@Tag(name = "AI Tools", description = "Dispatchable AI-backed tools.")
public class MathAuditorAgentController {
private static final Pattern NEWLINE_PATTERN = Pattern.compile("[\\r\\n]");
private final MathAuditorOrchestrator orchestrator;
private final AiFeatureGate aiFeatureGate;
@PostMapping(value = "/math-auditor-agent", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
@Operation(
@@ -77,15 +81,17 @@ public class MathAuditorAgentController {
+ " ignored (default: 0.01)")
@RequestParam(value = "tolerance", defaultValue = "0.01")
BigDecimal tolerance) {
aiFeatureGate.requireMathAuditor();
AiToolInputValidator.validatePdfUpload(fileInput);
if (tolerance.compareTo(BigDecimal.ZERO) < 0) {
return ResponseEntity.badRequest().build();
}
String originalFilename = fileInput.getOriginalFilename();
String safeName =
fileInput.getOriginalFilename() != null
? fileInput.getOriginalFilename().replaceAll("[\\r\\n]", "_")
originalFilename != null
? NEWLINE_PATTERN.matcher(originalFilename).replaceAll("_")
: "<unnamed>";
log.info("[math-auditor-agent] request file={} tolerance={}", safeName, tolerance);
app/proprietary/src/main/java/stirling/software/proprietary/controller/api/PdfCommentAgentController.java
+8 -2
View File
@@ -1,6 +1,7 @@
package stirling.software.proprietary.controller.api;
import java.io.IOException;
import java.util.regex.Pattern;
import org.springframework.core.io.ByteArrayResource;
import org.springframework.core.io.Resource;
@@ -20,6 +21,7 @@ import io.swagger.v3.oas.annotations.tags.Tag;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.proprietary.service.AiFeatureGate;
import stirling.software.proprietary.service.AiToolResponseHeaders;
import stirling.software.proprietary.service.PdfCommentAgentOrchestrator;
import stirling.software.proprietary.service.PdfCommentAgentOrchestrator.AnnotatedPdf;
@@ -45,8 +47,10 @@ import tools.jackson.databind.node.ObjectNode;
@Tag(name = "AI Tools", description = "Dispatchable AI-backed tools.")
public class PdfCommentAgentController {
private static final Pattern NEWLINE_PATTERN = Pattern.compile("[\\r\\n]");
private final PdfCommentAgentOrchestrator orchestrator;
private final ObjectMapper objectMapper;
private final AiFeatureGate aiFeatureGate;
@PostMapping(
value = "/pdf-comment-agent",
@@ -77,10 +81,12 @@ public class PdfCommentAgentController {
@RequestParam("prompt")
String prompt)
throws IOException {
aiFeatureGate.requirePdfComment();
String originalFilename = fileInput.getOriginalFilename();
String safeName =
fileInput.getOriginalFilename() != null
? fileInput.getOriginalFilename().replaceAll("[\\r\\n]", "_")
originalFilename != null
? NEWLINE_PATTERN.matcher(originalFilename).replaceAll("_")
: "<unnamed>";
log.info(
"[pdf-comment-agent] request file={} promptLen={}",
app/proprietary/src/main/java/stirling/software/proprietary/controller/api/PortalApiKeysController.java
+54
View File
@@ -0,0 +1,54 @@
package stirling.software.proprietary.controller.api;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.DeleteMapping;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestParam;
import io.swagger.v3.oas.annotations.Operation;
import lombok.RequiredArgsConstructor;
import stirling.software.common.annotations.api.ProprietaryUiDataApi;
import stirling.software.proprietary.model.api.apikey.CreateApiKeyRequest;
import stirling.software.proprietary.model.api.apikey.CreatedApiKeyDto;
import stirling.software.proprietary.model.api.apikey.PortalApiKeysResponse;
import stirling.software.proprietary.security.service.ApiKeyManagementService;
/**
* Real backing for the portal Infrastructure → API Keys tab: list/create/revoke named, personal API
* keys. Replaces the former portal-only mock endpoint. Not gated behind an Enterprise license - API
* keys are a core auth feature available on every self-hosted instance.
*/
@ProprietaryUiDataApi
@RequiredArgsConstructor
public class PortalApiKeysController {
private final ApiKeyManagementService apiKeyManagementService;
// tier accepted for endpoint symmetry with the other infra tabs; ignored here.
@GetMapping("/infrastructure/api-keys")
@Operation(summary = "List API keys", description = "The caller's personal API keys.")
public ResponseEntity<PortalApiKeysResponse> list(
@RequestParam(value = "tier", required = false) String tier) {
return ResponseEntity.ok(apiKeyManagementService.listVisibleKeys());
}
@PostMapping("/infrastructure/api-keys")
@Operation(
summary = "Create an API key",
description = "Mints a personal key and returns its one-time secret.")
public ResponseEntity<CreatedApiKeyDto> create(@RequestBody CreateApiKeyRequest request) {
return ResponseEntity.ok(apiKeyManagementService.createKey(request));
}
@DeleteMapping("/infrastructure/api-keys/{id}")
@Operation(summary = "Revoke an API key", description = "Disables a key the caller owns.")
public ResponseEntity<Void> revoke(@PathVariable("id") Long id) {
apiKeyManagementService.revokeKey(id);
return ResponseEntity.noContent().build();
}
}
app/proprietary/src/main/java/stirling/software/proprietary/controller/api/ProprietaryUIDataController.java
+112 -103
View File
@@ -3,7 +3,6 @@ package stirling.software.proprietary.controller.api;
import static stirling.software.common.util.ProviderUtils.validateProvider;
import java.time.Instant;
import java.time.temporal.ChronoUnit;
import java.util.*;
import java.util.stream.Collectors;
@@ -45,7 +44,6 @@ import stirling.software.proprietary.security.config.EnterpriseEndpoint;
import stirling.software.proprietary.security.database.repository.SessionRepository;
import stirling.software.proprietary.security.database.repository.UserRepository;
import stirling.software.proprietary.security.model.Authority;
import stirling.software.proprietary.security.model.SessionEntity;
import stirling.software.proprietary.security.model.User;
import stirling.software.proprietary.security.model.dto.AdminUserSummary;
import stirling.software.proprietary.security.repository.TeamMembershipRepository;
@@ -169,16 +167,8 @@ public class ProprietaryUIDataController {
boolean isFirstTimeSetup = false;
boolean showDefaultCredentials = false;
List<User> allUsers = userRepository.findAll();
List<User> realUsers =
allUsers.stream()
.filter(
user ->
!Role.INTERNAL_API_USER
.getRoleId()
.equals(user.getUsername()))
.toList();
long userCount = realUsers.size();
// Count real users, excluding the internal API user.
long userCount = userRepository.countByUsernameNot(Role.INTERNAL_API_USER.getRoleId());
if (userCount == 0) {
isFirstTimeSetup = true;
@@ -265,92 +255,67 @@ public class ProprietaryUIDataController {
@PreAuthorize("hasRole('ADMIN')")
@Operation(summary = "Get admin settings data")
public ResponseEntity<AdminSettingsData> getAdminSettingsData(Authentication authentication) {
List<User> allUsers = userRepository.findAllWithTeam();
Iterator<User> iterator = allUsers.iterator();
List<User> allUsers = userRepository.findAllWithTeamAndAuthorities();
Map<String, String> roleDetails = Role.getAllRoleDetails();
// Drop the internal API user and internal-team members; the roster never shows them.
boolean hasInternalApiUser = false;
List<User> visibleUsers = new ArrayList<>(allUsers.size());
for (User user : allUsers) {
if (user == null) {
continue;
}
if (isInternalApiUser(user)) {
hasInternalApiUser = true;
continue;
}
if (user.getTeam() != null
&& TeamService.INTERNAL_TEAM_NAME.equals(user.getTeam().getName())) {
continue;
}
visibleUsers.add(user);
}
if (hasInternalApiUser) {
roleDetails.remove(Role.INTERNAL_API_USER.getRoleId());
}
// All users' settings in one query (mfaSecret masked).
Map<Long, Map<String, String>> settingsByUserId =
loadSettingsByUserId(visibleUsers.stream().map(User::getId).toList());
// Active = any non-expired session within the inactivity window; expiry is left to
// SessionScheduled.
int maxInactiveInterval = sessionPersistentRegistry.getMaxInactiveInterval();
Instant activeCutoff = Instant.now().minusSeconds(maxInactiveInterval);
Map<String, Instant> lastRequestByPrincipal = new HashMap<>();
for (Object[] row : sessionRepository.findLatestRequestPerPrincipal()) {
if (row[0] != null) {
lastRequestByPrincipal.put((String) row[0], (Instant) row[1]);
}
}
Set<String> activePrincipals =
new HashSet<>(sessionRepository.findActivePrincipalsSince(activeCutoff));
Map<String, Boolean> userSessions = new HashMap<>();
Map<String, Date> userLastRequest = new HashMap<>();
Map<String, Map<String, String>> userSettings = new HashMap<>();
int activeUsers = 0;
int disabledUsers = 0;
while (iterator.hasNext()) {
User user = iterator.next();
if (user != null) {
String username = user.getUsername();
boolean shouldRemove = false;
// Check if user is an INTERNAL_API_USER
for (Authority authority : user.getAuthorities()) {
if (authority.getAuthority().equals(Role.INTERNAL_API_USER.getRoleId())) {
shouldRemove = true;
roleDetails.remove(Role.INTERNAL_API_USER.getRoleId());
break;
}
}
// Check if user is part of the Internal team
if (user.getTeam() != null
&& TeamService.INTERNAL_TEAM_NAME.equals(user.getTeam().getName())) {
shouldRemove = true;
}
if (shouldRemove) {
iterator.remove();
continue;
}
// Session status and last request time
int maxInactiveInterval = sessionPersistentRegistry.getMaxInactiveInterval();
boolean hasActiveSession = false;
Date lastRequest = null;
Optional<SessionEntity> latestSession =
sessionPersistentRegistry.findLatestSession(username);
if (latestSession.isPresent()) {
SessionEntity sessionEntity = latestSession.get();
Instant lastAccessedTime =
Optional.ofNullable(sessionEntity.getLastRequest())
.orElse(Instant.EPOCH);
Instant now = Instant.now();
Instant expirationTime =
lastAccessedTime.plus(maxInactiveInterval, ChronoUnit.SECONDS);
if (now.isAfter(expirationTime)) {
sessionPersistentRegistry.expireSession(sessionEntity.getSessionId());
} else {
hasActiveSession = !sessionEntity.isExpired();
}
lastRequest = Date.from(lastAccessedTime);
} else {
lastRequest = new Date(0);
}
User userWithSettings =
userRepository.findByIdWithSettings(user.getId()).orElse(user);
// Mask mfaSecret if present in settings
Map<String, String> originalSettings = userWithSettings.getSettings();
Map<String, String> settingsCopy =
originalSettings != null
? new HashMap<>(originalSettings)
: new HashMap<>();
if (settingsCopy.containsKey("mfaSecret")) {
settingsCopy.put("mfaSecret", "********");
}
userSettings.put(username, settingsCopy);
userSessions.put(username, hasActiveSession);
userLastRequest.put(username, lastRequest);
if (hasActiveSession) activeUsers++;
if (!user.isEnabled()) disabledUsers++;
}
for (User user : visibleUsers) {
String username = user.getUsername();
boolean hasActiveSession = activePrincipals.contains(username);
Instant lastRequest = lastRequestByPrincipal.get(username);
userSessions.put(username, hasActiveSession);
userLastRequest.put(
username, lastRequest != null ? Date.from(lastRequest) : new Date(0));
userSettings.put(username, maskSecrets(settingsByUserId.get(user.getId())));
if (hasActiveSession) activeUsers++;
if (!user.isEnabled()) disabledUsers++;
}
// Sort users by active status and last request date
List<User> sortedUsers =
allUsers.stream()
visibleUsers.stream()
.sorted(
(u1, u2) -> {
boolean u1Active = userSessions.get(u1.getUsername());
@@ -380,11 +345,30 @@ public class ProprietaryUIDataController {
int licenseMaxUsers = licenseSettingsService.getSettings().getLicenseMaxUsers();
boolean premiumEnabled = applicationProperties.getPremium().isEnabled();
// Convert User entities to AdminUserSummary DTOs to exclude sensitive fields
Set<Long> leaderUserIds = leaderUserIds();
// Resolve portal access for the whole roster. The teamLead display flag counts a
// LEADER membership on any team (mirrors /me), but the portal default policy only
// admits leaders of their own active team, so the bulk check gets the narrower set.
List<TeamMembership> leaderMemberships =
teamMembershipRepository.findByRoleFetchingUserAndTeam(TeamRole.LEADER);
Set<Long> leaderUserIds =
leaderMemberships.stream()
.map(row -> row.getUser().getId())
.collect(Collectors.toSet());
Set<Long> activeTeamLeaderUserIds =
leaderMemberships.stream()
.filter(
row ->
row.getUser().getTeam() != null
&& row.getTeam()
.getId()
.equals(row.getUser().getTeam().getId()))
.map(row -> row.getUser().getId())
.collect(Collectors.toSet());
Set<Long> portalAccessUserIds =
resourceAccessService.usersWithPortalAccess(sortedUsers, activeTeamLeaderUserIds);
List<AdminUserSummary> userSummaries =
sortedUsers.stream()
.map(user -> convertUserToSummary(user, leaderUserIds))
.map(user -> convertUserToSummary(user, leaderUserIds, portalAccessUserIds))
.toList();
AdminSettingsData data = new AdminSettingsData();
@@ -393,7 +377,7 @@ public class ProprietaryUIDataController {
data.setRoleDetails(roleDetails);
data.setUserSessions(userSessions);
data.setUserLastRequest(userLastRequest);
data.setTotalUsers(allUsers.size());
data.setTotalUsers(visibleUsers.size());
data.setActiveUsers(activeUsers);
data.setDisabledUsers(disabledUsers);
data.setTeams(allTeams);
@@ -516,7 +500,8 @@ public class ProprietaryUIDataController {
}
List<User> teamUsers = userRepository.findAllByTeamId(id);
List<User> allUsers = userRepository.findAllWithTeam();
// Fetch authorities + team for the available-users list.
List<User> allUsers = userRepository.findAllWithTeamAndAuthorities();
List<User> availableUsers =
allUsers.stream()
.filter(
@@ -568,24 +553,48 @@ public class ProprietaryUIDataController {
return ResponseEntity.ok(data);
}
/** User ids holding a LEADER membership on any team. */
private Set<Long> leaderUserIds() {
return teamMembershipRepository.findByRoleFetchingUserAndTeam(TeamRole.LEADER).stream()
.map(row -> row.getUser().getId())
.collect(Collectors.toSet());
/** Whether the user holds the internal-API authority (never shown in the roster). */
private boolean isInternalApiUser(User user) {
for (Authority authority : user.getAuthorities()) {
if (Role.INTERNAL_API_USER.getRoleId().equals(authority.getAuthority())) {
return true;
}
}
return false;
}
/** Assemble per-user settings maps from the flat (id, key, value) rows of one bulk query. */
private Map<Long, Map<String, String>> loadSettingsByUserId(List<Long> userIds) {
Map<Long, Map<String, String>> byUser = new HashMap<>();
if (userIds.isEmpty()) {
return byUser;
}
for (Object[] row : userRepository.findSettingsByUserIds(userIds)) {
byUser.computeIfAbsent((Long) row[0], id -> new HashMap<>())
.put((String) row[1], (String) row[2]);
}
return byUser;
}
/** Copy a settings map with mfaSecret masked; null-safe. */
private Map<String, String> maskSecrets(Map<String, String> settings) {
Map<String, String> copy = settings != null ? new HashMap<>(settings) : new HashMap<>();
if (copy.containsKey("mfaSecret")) {
copy.put("mfaSecret", "********");
}
return copy;
}
/**
* Convert User entity to AdminUserSummary DTO, excluding sensitive fields like password and
* apiKey.
* Convert a User to AdminUserSummary (excludes sensitive fields); portal access is passed in.
*/
private AdminUserSummary convertUserToSummary(User user, Set<Long> leaderUserIds) {
private AdminUserSummary convertUserToSummary(
User user, Set<Long> leaderUserIds, Set<Long> portalAccessUserIds) {
AdminUserSummary summary = new AdminUserSummary();
summary.setId(user.getId());
summary.setTeamLead(leaderUserIds.contains(user.getId()));
// Authoritative portal access, same call /me uses, so the roster honors the configured
// policy instead of the frontend guessing from role/team-leadership.
summary.setPortalAccess(resourceAccessService.canAccessPortal(user));
// Portal access (same policy /me uses).
summary.setPortalAccess(portalAccessUserIds.contains(user.getId()));
summary.setUsername(user.getUsername());
summary.setEmail(user.getUsername()); // Use username as email for consistency
summary.setRoleName(user.getRoleName());
app/proprietary/src/main/java/stirling/software/proprietary/integration/api/ApiAuthType.java
+25
View File
@@ -0,0 +1,25 @@
package stirling.software.proprietary.integration.api;
/**
* How an {@link stirling.software.proprietary.integration.model.IntegrationType#API} connection
* authenticates.
*/
public enum ApiAuthType {
/** No credentials; the endpoint is open or authorises by network position. */
NONE,
/** {@code Authorization: Bearer <token>}. */
BEARER,
/** {@code Authorization: Basic base64(username:password)}. */
BASIC,
/** The token in a caller-named header, e.g. {@code X-API-Key: <token>}. */
HEADER,
/**
* The connection logs in first and reuses the short-lived token it gets back.
*
* <p>For the large class of enterprise APIs - ConsignO Cloud, OAuth2 client-credentials, and
* others - where credentials buy a token rather than authenticating a call directly. Without
* this a step could not reach them at all: each call needs a token, and a stateless step has
* nowhere to obtain or keep one. See {@link ApiTokenLogin}.
*/
TOKEN_LOGIN
}
app/proprietary/src/main/java/stirling/software/proprietary/integration/api/ApiConnectionResolver.java
+130
View File
@@ -0,0 +1,130 @@
package stirling.software.proprietary.integration.api;
import java.util.LinkedHashMap;
import java.util.Map;
import org.springframework.security.core.Authentication;
import org.springframework.security.core.context.SecurityContextHolder;
import org.springframework.security.core.userdetails.UserDetails;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import lombok.RequiredArgsConstructor;
import stirling.software.proprietary.access.model.ResourceType;
import stirling.software.proprietary.access.service.OwnershipService;
import stirling.software.proprietary.integration.model.IntegrationConfig;
import stirling.software.proprietary.integration.model.IntegrationType;
import stirling.software.proprietary.integration.repository.IntegrationConfigRepository;
import stirling.software.proprietary.security.model.User;
import stirling.software.proprietary.security.service.UserService;
import tools.jackson.core.type.TypeReference;
import tools.jackson.databind.ObjectMapper;
/**
* Dereferences a step's {@code connectionId} to a stored integration config.
*
* <p>Mirrors {@code S3ConnectionResolver}. When an authenticated caller is present the connection
* must be usable by them; a background worker thread carries no {@code SecurityContext} and skips
* that check, relying on the step having been access-checked when the policy was saved or when an
* ad-hoc run was dispatched - see {@link IntegrationStepValidator}, which is what makes that
* assumption true rather than merely hoped for.
*/
@Service
@RequiredArgsConstructor
@Transactional(readOnly = true)
public class ApiConnectionResolver {
private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper();
private final IntegrationConfigRepository connections;
private final OwnershipService ownership;
private final UserService userService;
/** The raw config map for a connection of the given type. */
public Map<String, Object> resolveConfig(Long connectionId, IntegrationType type) {
IntegrationConfig connection =
connections
.findById(connectionId)
.filter(cfg -> cfg.getIntegrationType() == type)
.filter(this::usableByCurrentUser)
// Existence and access collapse into one error so a caller cannot tell
// "no such connection" from "someone else's connection" and enumerate ids.
.orElseThrow(
() ->
new IllegalArgumentException(
"unknown or inaccessible "
+ type.name().toLowerCase()
+ " connection"));
if (!connection.isEnabled()) {
throw new IllegalArgumentException(
type.name().toLowerCase() + " connection is disabled");
}
return configOf(connection);
}
/** The settings for a generic {@code API} connection. */
public ApiConnectionSettings resolve(Long connectionId) {
return ApiConnectionSettings.from(resolveConfig(connectionId, IntegrationType.API));
}
/** Parse a {@code connectionId} step parameter; null when absent. */
public static Long connectionId(Object reference) {
if (reference == null || (reference instanceof String s && s.isBlank())) {
return null;
}
if (reference instanceof Number number) {
return number.longValue();
}
try {
return Long.valueOf(reference.toString().trim());
} catch (NumberFormatException e) {
throw new IllegalArgumentException(
"'connectionId' is not a valid connection reference: " + reference);
}
}
/**
* Whether the current caller may use this connection. A missing principal means a worker
* thread, where access was established earlier; it must never be the only thing standing
* between a caller and a connection, or the check becomes a confused deputy.
*/
private boolean usableByCurrentUser(IntegrationConfig connection) {
User user = currentUser();
return user == null || ownership.canUse(ResourceType.INTEGRATION_CONFIG, connection, user);
}
// Mirrors ResourceAccessSecurity's principal resolution; null when unauthenticated.
private User currentUser() {
Authentication auth = SecurityContextHolder.getContext().getAuthentication();
if (auth == null || !auth.isAuthenticated()) {
return null;
}
Object principal = auth.getPrincipal();
if (principal instanceof User user) {
return user;
}
if (principal instanceof UserDetails userDetails) {
return userService.findByUsername(userDetails.getUsername()).orElse(null);
}
if (principal instanceof String username && !"anonymousUser".equals(username)) {
return userService.findByUsername(username).orElse(null);
}
return null;
}
private static Map<String, Object> configOf(IntegrationConfig connection) {
String json = connection.getConfig();
if (json == null || json.isBlank()) {
return Map.of();
}
try {
return OBJECT_MAPPER.readValue(
json, new TypeReference<LinkedHashMap<String, Object>>() {});
} catch (Exception e) {
throw new IllegalArgumentException(
"connection '" + connection.getName() + "' has unreadable config", e);
}
}
}
app/proprietary/src/main/java/stirling/software/proprietary/integration/api/ApiConnectionSettings.java
+282
View File
@@ -0,0 +1,282 @@
package stirling.software.proprietary.integration.api;
import java.net.URI;
import java.net.URISyntaxException;
import java.util.LinkedHashMap;
import java.util.Locale;
import java.util.Map;
import java.util.Objects;
import java.util.Set;
/**
* A resolved {@code API} connection: where to call, and how to authenticate.
*
* <p>{@code baseUrl} is the security anchor of the whole feature. It is set by whoever can manage
* the connection (an admin or team owner) and is the only thing that decides which host is
* contacted. A pipeline step supplies a <em>relative path</em> only, resolved under this base by
* {@link ExternalApiPaths}, so a step author can never pivot the call to a host of their choosing.
* Widening that - letting a step pass a full URL - would turn every policy into an SSRF primitive.
*
* <p>Whether the base URL may resolve to a private address is deliberately <em>not</em> a field
* here. Any user may create an API connection (unlike S3, which {@code IntegrationConfigService}
* restricts to admins), so a per-connection opt-in would let a user grant themselves a fetch of the
* cloud metadata service. It is an operator property instead - {@code
* policies.allowPrivateApiEndpoints} - checked by {@link ApiIntegrationValidator}.
*/
public record ApiConnectionSettings(
String baseUrl,
ApiAuthType authType,
String headerName,
String headerPrefix,
String token,
String username,
String password,
Map<String, String> headers,
ApiTokenLogin tokenLogin,
Set<String> resultUrlHosts,
int timeoutSeconds) {
static final String BASE_URL_OPTION = "baseUrl";
static final String AUTH_TYPE_OPTION = "authType";
static final String HEADER_NAME_OPTION = "headerName";
static final String HEADER_PREFIX_OPTION = "headerPrefix";
// "token"/"password" contain SecretMasker hints, so they mask on read and merge on update.
static final String TOKEN_OPTION = "token";
static final String USERNAME_OPTION = "username";
static final String PASSWORD_OPTION = "password";
static final String HEADERS_OPTION = "headers";
static final String RESULT_URL_HOSTS_OPTION = "resultUrlHosts";
static final String TIMEOUT_SECONDS_OPTION = "timeoutSeconds";
static final int DEFAULT_TIMEOUT_SECONDS = 60;
private static final int MAX_TIMEOUT_SECONDS = 600;
public ApiConnectionSettings {
headers = headers == null ? Map.of() : Map.copyOf(headers);
resultUrlHosts = resultUrlHosts == null ? Set.of() : Set.copyOf(resultUrlHosts);
}
/**
* @throws IllegalArgumentException if the config is unusable; the message is surfaced to the
* operator editing the connection, so it names the offending option.
*/
public static ApiConnectionSettings from(Map<String, Object> options) {
String baseUrl = trimmed(options.get(BASE_URL_OPTION));
if (baseUrl == null) {
throw new IllegalArgumentException("api config requires a 'baseUrl' option");
}
URI uri = parseHttpUrl(baseUrl);
if (uri.getQuery() != null || uri.getFragment() != null) {
throw new IllegalArgumentException(
"api config 'baseUrl' must not carry a query string or fragment");
}
ApiAuthType authType = parseAuthType(trimmed(options.get(AUTH_TYPE_OPTION)));
String headerName = trimmed(options.get(HEADER_NAME_OPTION));
// Many APIs want a scheme before the token ("Authorization: Token abc",
// "Authorization: DeepL-Auth-Key abc"). Without this a preset would have to make the
// operator paste the scheme into the secret itself, which reads as a typo waiting to
// happen.
String headerPrefix = trimmed(options.get(HEADER_PREFIX_OPTION));
String token = trimmed(options.get(TOKEN_OPTION));
String username = trimmed(options.get(USERNAME_OPTION));
String password = trimmed(options.get(PASSWORD_OPTION));
switch (authType) {
case BEARER -> require(token, "api config authType 'BEARER' requires a 'token'");
case HEADER -> {
require(token, "api config authType 'HEADER' requires a 'token'");
require(headerName, "api config authType 'HEADER' requires a 'headerName'");
if (!ExternalApiHeaders.isValidName(headerName)) {
throw new IllegalArgumentException(
"api config 'headerName' is not a valid HTTP header name: "
+ headerName);
}
}
case BASIC -> {
require(username, "api config authType 'BASIC' requires a 'username'");
require(password, "api config authType 'BASIC' requires a 'password'");
}
case TOKEN_LOGIN -> {
/* validated by ApiTokenLogin.from below */
}
case NONE -> {
/* nothing to check */
}
}
return new ApiConnectionSettings(
stripTrailingSlash(baseUrl),
authType,
headerName,
headerPrefix,
token,
username,
password,
parseHeaders(options.get(HEADERS_OPTION)),
authType == ApiAuthType.TOKEN_LOGIN ? ApiTokenLogin.from(options) : null,
parseResultUrlHosts(options.get(RESULT_URL_HOSTS_OPTION)),
parseTimeout(options.get(TIMEOUT_SECONDS_OPTION)));
}
/** The configured base as a URI; callers resolve step paths under it. */
public URI baseUri() {
return URI.create(baseUrl);
}
/**
* Identity of this connection's login for token-cache purposes. Includes the credentials, so
* editing a password evicts the token cached under the old one rather than reusing it until it
* expires.
*/
String tokenCacheKey() {
return baseUrl + "|" + Objects.hash(tokenLogin);
}
private static URI parseHttpUrl(String value) {
URI uri;
try {
uri = new URI(value);
} catch (URISyntaxException e) {
throw new IllegalArgumentException("api config 'baseUrl' is not a valid URL", e);
}
String scheme = uri.getScheme() == null ? null : uri.getScheme().toLowerCase(Locale.ROOT);
if (!"http".equals(scheme) && !"https".equals(scheme)) {
throw new IllegalArgumentException(
"api config 'baseUrl' must be an http(s) URL, e.g. https://api.example.com");
}
if (uri.getHost() == null || uri.getHost().isBlank()) {
throw new IllegalArgumentException("api config 'baseUrl' must include a host");
}
return uri;
}
private static ApiAuthType parseAuthType(String value) {
if (value == null) {
return ApiAuthType.NONE;
}
try {
return ApiAuthType.valueOf(value.toUpperCase(Locale.ROOT));
} catch (IllegalArgumentException e) {
throw new IllegalArgumentException(
"api config 'authType' must be one of NONE, BEARER, BASIC, HEADER; got "
+ value);
}
}
/** Static headers sent on every call. Rejects anything auth-bearing to keep one auth path. */
private static Map<String, String> parseHeaders(Object value) {
if (value == null) {
return Map.of();
}
if (!(value instanceof Map<?, ?> raw)) {
throw new IllegalArgumentException("api config 'headers' must be an object");
}
Map<String, String> headers = new LinkedHashMap<>();
for (Map.Entry<?, ?> entry : raw.entrySet()) {
String name = trimmed(entry.getKey());
String headerValue = entry.getValue() == null ? null : entry.getValue().toString();
if (name == null) {
continue;
}
if (!ExternalApiHeaders.isValidName(name)) {
throw new IllegalArgumentException(
"api config 'headers' has an invalid header name: " + name);
}
if (ExternalApiHeaders.isReserved(name)) {
throw new IllegalArgumentException(
"api config 'headers' must not set '"
+ name
+ "'; use 'authType' and 'token' instead");
}
if (headerValue == null || !ExternalApiHeaders.isValidValue(headerValue)) {
throw new IllegalArgumentException(
"api config 'headers' has an invalid value for '" + name + "'");
}
headers.put(name, headerValue);
}
return headers;
}
/**
* Hosts a result may be fetched from, beyond the connection's own. Declared by the operator
* because the alternative - trusting the host named in the API's response - is an SSRF.
*/
private static Set<String> parseResultUrlHosts(Object value) {
if (value == null) {
return Set.of();
}
if (!(value instanceof java.util.List<?> list)) {
throw new IllegalArgumentException(
"api config 'resultUrlHosts' must be a list of hostnames");
}
Set<String> out = new java.util.LinkedHashSet<>();
for (Object entry : list) {
String host = trimmed(entry);
if (host == null) {
continue;
}
if (host.contains("/") || host.contains(":") || host.contains("*")) {
// A URL, port or wildcard here would read as broader than it is; subdomains are
// already covered by the "endsWith('.' + host)" rule at match time.
throw new IllegalArgumentException(
"api config 'resultUrlHosts' takes bare hostnames, e.g."
+ " cdn.vendor.com; got "
+ host);
}
out.add(host.toLowerCase(Locale.ROOT));
}
return out;
}
private static int parseTimeout(Object value) {
if (value == null) {
return DEFAULT_TIMEOUT_SECONDS;
}
int seconds;
try {
seconds = Integer.parseInt(value.toString().trim());
} catch (NumberFormatException e) {
throw new IllegalArgumentException("api config 'timeoutSeconds' must be a number");
}
if (seconds < 1 || seconds > MAX_TIMEOUT_SECONDS) {
throw new IllegalArgumentException(
"api config 'timeoutSeconds' must be between 1 and " + MAX_TIMEOUT_SECONDS);
}
return seconds;
}
private static void require(String value, String message) {
if (value == null) {
throw new IllegalArgumentException(message);
}
}
private static String stripTrailingSlash(String value) {
String out = value;
while (out.endsWith("/")) {
out = out.substring(0, out.length() - 1);
}
return out;
}
private static String trimmed(Object value) {
if (value == null) {
return null;
}
String text = value.toString().trim();
return text.isEmpty() ? null : text;
}
/** Never prints the credentials, so an accidental log line cannot leak them. */
@Override
public String toString() {
return "ApiConnectionSettings[baseUrl="
+ baseUrl
+ ", authType="
+ authType
+ ", timeoutSeconds="
+ timeoutSeconds
+ "]";
}
}
app/proprietary/src/main/java/stirling/software/proprietary/integration/api/ApiIntegrationValidator.java
+109
View File
@@ -0,0 +1,109 @@
package stirling.software.proprietary.integration.api;
import java.util.Map;
import org.springframework.stereotype.Component;
import lombok.RequiredArgsConstructor;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.proprietary.cluster.s3.S3Clients;
import stirling.software.proprietary.integration.model.IntegrationType;
import stirling.software.proprietary.integration.service.IntegrationConfigValidator;
/**
* The {@code API} connection schema, enforced when the config is saved: an http(s) base URL, a
* coherent auth block, and a host that must not reach private addresses without the operator
* opt-in.
*
* <p>The host check runs here so a bad connection fails in the form rather than mid-run. It is not
* the only check - {@link ExternalApiCaller} re-checks before dispatch, because DNS can be
* re-pointed at a private address long after save time (a check-then-use gap this validator alone
* cannot close).
*/
@Component
@RequiredArgsConstructor
public class ApiIntegrationValidator implements IntegrationConfigValidator {
private final ApplicationProperties applicationProperties;
@Override
public IntegrationType type() {
return IntegrationType.API;
}
@Override
public void validate(Map<String, Object> config) {
ApiConnectionSettings settings = ApiConnectionSettings.from(config);
requirePublicHost(settings, applicationProperties, "API connection base URL");
}
/**
* Shared by every integration type that dials an operator-supplied host, so they cannot drift
* apart on what counts as reachable.
*/
static void requirePublicHost(
ApiConnectionSettings settings,
ApplicationProperties applicationProperties,
String settingName) {
// Block the cloud metadata service unconditionally - before the opt-in check. The private-
// endpoint opt-in exists for on-prem services (RFC1918, an internal gateway), but the
// metadata endpoint is never a real integration and reaching it is the highest-value SSRF:
// it hands out the instance's own IAM credentials. So it stays blocked even when the
// operator has allowed private endpoints.
denyCloudMetadata(settings.baseUri(), settingName);
try {
S3Clients.validateEndpointHost(
settings.baseUri(),
applicationProperties.getPolicies().isAllowPrivateApiEndpoints(),
settingName,
"set policies.allowPrivateApiEndpoints=true to opt in (e.g. for an on-prem"
+ " integration).");
} catch (IllegalStateException e) {
throw new IllegalArgumentException(e.getMessage(), e);
}
}
/** AWS/GCP/Azure, Oracle and IBM metadata addresses; mirrors {@code SsrfProtectionService}. */
private static final java.util.Set<String> CLOUD_METADATA_IPS =
java.util.Set.of(
"169.254.169.254", "169.254.169.253", "169.254.169.250", "fd00:ec2::254");
private static void denyCloudMetadata(java.net.URI uri, String settingName) {
String host = uri.getHost();
if (host == null || host.isBlank()) {
return; // a missing host is S3Clients' error to report, with its own message
}
java.net.InetAddress[] addresses;
try {
addresses = java.net.InetAddress.getAllByName(host);
} catch (java.net.UnknownHostException e) {
return; // an unresolvable host is likewise left to S3Clients to reject
}
for (java.net.InetAddress address : addresses) {
String ip = normalise(address.getHostAddress());
if (CLOUD_METADATA_IPS.stream().anyMatch(ip::startsWith)) {
throw new IllegalArgumentException(
settingName
+ " host '"
+ host
+ "' resolves to the cloud metadata service ("
+ ip
+ "), which is never a valid integration target.");
}
}
}
/** Strip an IPv4-mapped-IPv6 prefix and any zone id so the compare sees a bare address. */
private static String normalise(String ip) {
String out = ip;
int zone = out.indexOf('%');
if (zone >= 0) {
out = out.substring(0, zone);
}
if (out.startsWith("::ffff:")) {
out = out.substring(7);
}
return out;
}
}
app/proprietary/src/main/java/stirling/software/proprietary/integration/api/ApiTokenCache.java
+149
View File
@@ -0,0 +1,149 @@
package stirling.software.proprietary.integration.api;
import java.io.IOException;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.time.Duration;
import java.util.Map;
import java.util.concurrent.TimeUnit;
import org.springframework.http.MediaType;
import com.github.benmanes.caffeine.cache.Cache;
import com.github.benmanes.caffeine.cache.Caffeine;
import lombok.extern.slf4j.Slf4j;
import tools.jackson.databind.ObjectMapper;
/**
* Obtains and caches the short-lived tokens of {@link ApiAuthType#TOKEN_LOGIN} connections.
*
* <p>The step that uses a token is stateless and runs once per document, so without a cache a
* hundred-document policy would perform a hundred logins - which many vendors rate-limit, and some
* treat as suspicious. The cache is keyed on the connection's login identity (credentials included)
* so that editing a password does not keep reusing the token bought with the old one.
*
* <p>Entries expire well inside the vendor's stated lifetime, and a 401 additionally evicts and
* retries once ({@link ExternalApiCaller}), so a token that expires early - or is revoked - costs
* one retry rather than a failed run.
*/
@Slf4j
public class ApiTokenCache {
/** Bounded so a deployment with many connections cannot grow this without limit. */
private static final int MAX_ENTRIES = 500;
private final Cache<String, String> tokens;
private final HttpClient httpClient;
private final ObjectMapper objectMapper;
ApiTokenCache(HttpClient httpClient, ObjectMapper objectMapper) {
this.httpClient = httpClient;
this.objectMapper = objectMapper;
this.tokens =
Caffeine.newBuilder()
.maximumSize(MAX_ENTRIES)
// Per-entry, because each connection states its own lifetime.
.expireAfter(
new com.github.benmanes.caffeine.cache.Expiry<String, String>() {
@Override
public long expireAfterCreate(
String key, String value, long currentTime) {
return ttlNanos(key);
}
@Override
public long expireAfterUpdate(
String key,
String value,
long currentTime,
long currentDuration) {
return ttlNanos(key);
}
@Override
public long expireAfterRead(
String key,
String value,
long currentTime,
long currentDuration) {
// Reading must not extend a token's life: the vendor's
// clock is running regardless of how often we use it.
return currentDuration;
}
})
.build();
}
// The TTL travels in the key so the Expiry callbacks can see it without a second lookup.
private static long ttlNanos(String key) {
int seconds = Integer.parseInt(key.substring(key.lastIndexOf('#') + 1));
return TimeUnit.SECONDS.toNanos(seconds);
}
/**
* The connection's current token, logging in if there is not a live one.
*
* @throws IOException if the login call fails or returns no token
*/
String token(ApiConnectionSettings settings) throws IOException {
String key = cacheKey(settings);
String cached = tokens.getIfPresent(key);
if (cached != null) {
return cached;
}
String token = login(settings);
tokens.put(key, token);
return token;
}
/** Drop the cached token, e.g. after a 401 says it is no longer accepted. */
void invalidate(ApiConnectionSettings settings) {
tokens.invalidate(cacheKey(settings));
}
private static String cacheKey(ApiConnectionSettings settings) {
return settings.tokenCacheKey() + "#" + settings.tokenLogin().tokenTtlSeconds();
}
private String login(ApiConnectionSettings settings) throws IOException {
ApiTokenLogin login = settings.tokenLogin();
URI target = ExternalApiPaths.resolve(settings.baseUri(), login.loginPath());
HttpRequest.Builder request =
HttpRequest.newBuilder(target)
.timeout(Duration.ofSeconds(settings.timeoutSeconds()))
.header("Content-Type", MediaType.APPLICATION_JSON_VALUE)
.POST(
HttpRequest.BodyPublishers.ofByteArray(
objectMapper.writeValueAsBytes(login.loginBody())));
login.loginHeaders().forEach(request::header);
ExternalApiCaller.Response response =
ExternalApiCaller.send(httpClient, request.build(), target);
if (!response.isSuccess()) {
// Deliberately does not echo the body: a login failure response can repeat the
// credentials back, and this message reaches the run log.
throw new IOException(
"Login to "
+ target.getHost()
+ login.loginPath()
+ " returned HTTP "
+ response.status());
}
try {
String token = login.extractToken(response, objectMapper);
log.debug("[external-api] obtained a token from {}", target.getHost());
return token;
} catch (IllegalStateException e) {
throw new IOException(e.getMessage(), e);
}
}
/** The auth header for an authenticated call. */
Map.Entry<String, String> authHeader(ApiConnectionSettings settings) throws IOException {
return settings.tokenLogin().authHeader(token(settings));
}
}
app/proprietary/src/main/java/stirling/software/proprietary/integration/api/ApiTokenLogin.java
+209
View File
@@ -0,0 +1,209 @@
package stirling.software.proprietary.integration.api;
import java.util.LinkedHashMap;
import java.util.Map;
import tools.jackson.databind.JsonNode;
import tools.jackson.databind.ObjectMapper;
/**
* How a connection turns credentials into a short-lived token.
*
* <p>Modelled on what real APIs actually do rather than on one vendor. The two axes that vary are
* where the token comes back ({@code tokenResponseHeader} or {@code tokenResponseJsonPath}) and how
* it is then presented ({@code tokenHeaderName} + {@code tokenPrefix}). That covers both ends of
* the spectrum:
*
* <ul>
* <li>ConsignO Cloud - {@code POST /auth/login} with {@code X-Client-Id}/{@code X-Client-Secret}
* headers and a {@code {username, password, tenantId}} body, returning the token in the
* {@code X-Auth-Token} <em>response header</em>, which is then sent back as {@code
* X-Auth-Token}.
* <li>OAuth2 client-credentials - a form or JSON post returning {@code {"access_token": ...}} in
* the body, sent back as {@code Authorization: Bearer ...}.
* </ul>
*
* <p>{@code loginBody} and {@code loginHeaders} are stored as nested maps rather than a
* pre-rendered JSON string so {@code SecretMasker} can recurse and mask the {@code password} /
* {@code X-Client-Secret} entries inside them. A flat string would sail past it and hand the
* password back in plaintext on every read of the connection.
*/
record ApiTokenLogin(
String loginPath,
Map<String, Object> loginBody,
Map<String, String> loginHeaders,
String tokenResponseHeader,
String tokenResponseJsonPath,
String tokenHeaderName,
String tokenPrefix,
int tokenTtlSeconds) {
static final String LOGIN_PATH_OPTION = "loginPath";
static final String LOGIN_BODY_OPTION = "loginBody";
static final String LOGIN_HEADERS_OPTION = "loginHeaders";
static final String TOKEN_RESPONSE_HEADER_OPTION = "tokenResponseHeader";
static final String TOKEN_RESPONSE_JSON_PATH_OPTION = "tokenResponseJsonPath";
static final String TOKEN_HEADER_NAME_OPTION = "tokenHeaderName";
static final String TOKEN_PREFIX_OPTION = "tokenPrefix";
static final String TOKEN_TTL_SECONDS_OPTION = "tokenTtlSeconds";
/**
* Conservative default. ConsignO's token lasts 30 minutes; caching for 25 leaves room for a
* slow call to finish on a token that was still valid when it started. A cache that expired
* exactly on the vendor's boundary would fail intermittently and look like a network fault.
*/
static final int DEFAULT_TOKEN_TTL_SECONDS = 1500;
private static final int MAX_TOKEN_TTL_SECONDS = 86400;
ApiTokenLogin {
loginBody = loginBody == null ? Map.of() : Map.copyOf(loginBody);
loginHeaders = loginHeaders == null ? Map.of() : Map.copyOf(loginHeaders);
}
static ApiTokenLogin from(Map<String, Object> options) {
String loginPath = trimmed(options.get(LOGIN_PATH_OPTION));
if (loginPath == null) {
throw new IllegalArgumentException(
"api config authType 'TOKEN_LOGIN' requires a 'loginPath', e.g. /auth/login");
}
String responseHeader = trimmed(options.get(TOKEN_RESPONSE_HEADER_OPTION));
String responseJsonPath = trimmed(options.get(TOKEN_RESPONSE_JSON_PATH_OPTION));
if ((responseHeader == null) == (responseJsonPath == null)) {
throw new IllegalArgumentException(
"api config authType 'TOKEN_LOGIN' needs exactly one of"
+ " 'tokenResponseHeader' (e.g. X-Auth-Token) or"
+ " 'tokenResponseJsonPath' (e.g. access_token) to say where the token"
+ " comes back");
}
String tokenHeaderName = trimmed(options.get(TOKEN_HEADER_NAME_OPTION));
if (tokenHeaderName == null) {
throw new IllegalArgumentException(
"api config authType 'TOKEN_LOGIN' requires a 'tokenHeaderName' saying which"
+ " header carries the token back, e.g. X-Auth-Token or Authorization");
}
if (!ExternalApiHeaders.isValidName(tokenHeaderName)) {
throw new IllegalArgumentException(
"api config 'tokenHeaderName' is not a valid HTTP header name: "
+ tokenHeaderName);
}
if (responseHeader != null && !ExternalApiHeaders.isValidName(responseHeader)) {
throw new IllegalArgumentException(
"api config 'tokenResponseHeader' is not a valid HTTP header name: "
+ responseHeader);
}
return new ApiTokenLogin(
loginPath,
nestedObject(options.get(LOGIN_BODY_OPTION), LOGIN_BODY_OPTION),
loginHeaders(options.get(LOGIN_HEADERS_OPTION)),
responseHeader,
responseJsonPath,
tokenHeaderName,
trimmed(options.get(TOKEN_PREFIX_OPTION)) == null
? ""
: trimmed(options.get(TOKEN_PREFIX_OPTION)) + " ",
ttl(options.get(TOKEN_TTL_SECONDS_OPTION)));
}
/** Pull the token out of a login response. */
String extractToken(ExternalApiCaller.Response response, ObjectMapper objectMapper) {
if (tokenResponseHeader != null) {
String value = response.header(tokenResponseHeader);
if (value == null || value.isBlank()) {
throw new IllegalStateException(
"Login succeeded but returned no '"
+ tokenResponseHeader
+ "' response header");
}
return value;
}
JsonNode node = response.bodyAsJson(objectMapper);
for (String segment : tokenResponseJsonPath.split("\\.")) {
if (node == null) {
break;
}
node = node.get(segment);
}
if (node == null || !node.isValueNode() || node.asString().isBlank()) {
throw new IllegalStateException(
"Login succeeded but its body had no token at '" + tokenResponseJsonPath + "'");
}
return node.asString();
}
/** The header to send on an authenticated call. */
Map.Entry<String, String> authHeader(String token) {
return Map.entry(tokenHeaderName, tokenPrefix + token);
}
private static Map<String, Object> nestedObject(Object value, String option) {
if (value == null) {
return Map.of();
}
if (!(value instanceof Map<?, ?> raw)) {
throw new IllegalArgumentException("api config '" + option + "' must be an object");
}
Map<String, Object> out = new LinkedHashMap<>();
raw.forEach((key, entry) -> out.put(String.valueOf(key), entry));
return out;
}
private static Map<String, String> loginHeaders(Object value) {
Map<String, String> out = new LinkedHashMap<>();
nestedObject(value, LOGIN_HEADERS_OPTION)
.forEach(
(name, entry) -> {
String headerValue = entry == null ? null : entry.toString();
if (!ExternalApiHeaders.isValidName(name)) {
throw new IllegalArgumentException(
"api config 'loginHeaders' has an invalid header name: "
+ name);
}
if (headerValue == null
|| !ExternalApiHeaders.isValidValue(headerValue)) {
throw new IllegalArgumentException(
"api config 'loginHeaders' has an invalid value for '"
+ name
+ "'");
}
out.put(name, headerValue);
});
return out;
}
private static int ttl(Object value) {
if (value == null) {
return DEFAULT_TOKEN_TTL_SECONDS;
}
int seconds;
try {
seconds = Integer.parseInt(value.toString().trim());
} catch (NumberFormatException e) {
throw new IllegalArgumentException("api config 'tokenTtlSeconds' must be a number");
}
if (seconds < 1 || seconds > MAX_TOKEN_TTL_SECONDS) {
throw new IllegalArgumentException(
"api config 'tokenTtlSeconds' must be between 1 and " + MAX_TOKEN_TTL_SECONDS);
}
return seconds;
}
private static String trimmed(Object value) {
if (value == null) {
return null;
}
String text = value.toString().trim();
return text.isEmpty() ? null : text;
}
/** Never prints the login body or headers: both carry the credentials. */
@Override
public String toString() {
return "ApiTokenLogin[loginPath="
+ loginPath
+ ", tokenTtlSeconds="
+ tokenTtlSeconds
+ "]";
}
}
app/proprietary/src/main/java/stirling/software/proprietary/integration/api/DocumentContext.java
+180
View File
@@ -0,0 +1,180 @@
package stirling.software.proprietary.integration.api;
import java.io.IOException;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.time.Instant;
import java.util.Base64;
import java.util.Calendar;
import java.util.HexFormat;
import java.util.List;
import java.util.Locale;
import org.apache.pdfbox.Loader;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDDocumentInformation;
import org.springframework.web.multipart.MultipartFile;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.service.PdfMetadataService;
import stirling.software.proprietary.integration.purview.PdfSensitivityLabels;
import stirling.software.proprietary.integration.purview.SensitivityLabel;
import tools.jackson.databind.JsonNode;
import tools.jackson.databind.ObjectMapper;
import tools.jackson.databind.node.ObjectNode;
/**
* Everything Stirling already knows about the document and the run, as one JSON object.
*
* <p>An external API almost always wants more than the bytes: what the file is, what it was called,
* whether it is already classified or labelled, and which policy sent it. All of that is in hand at
* the moment of the call, so it is offered rather than left for the operator to re-derive - most
* usefully the Purview label and the classifier's verdict, which turn a call-out into something the
* receiving system can make a decision with.
*
* <p>The shape is also the namespace for placeholders (see {@link Placeholders}), so {@code
* {{document.sha256}}} or {@code {{sensitivityLabel.name}}} in a field, path, or header resolves
* against exactly what is documented here:
*
* <pre>
* document.filename | .extension | .contentType | .sizeBytes | .sha256 | .base64
* .pageCount | .encrypted | .title | .author | .subject | .keywords
* .creator | .producer | .created | .modified
* classification.* the classifier policy's verdict, when it has run
* sensitivityLabel.labelId | .name | .siteId | .method | .protected
* run.policyName | .runId | .timestamp
* </pre>
*
* <p>Every field is best-effort: a non-PDF, an unparseable PDF, or an ad-hoc run with no policy
* simply omits what it cannot know. Building the context must never be the reason a step fails.
*/
@Slf4j
final class DocumentContext {
private DocumentContext() {}
static ObjectNode build(
MultipartFile file,
byte[] content,
String policyName,
String runId,
ObjectMapper objectMapper) {
ObjectNode root = objectMapper.createObjectNode();
ObjectNode document = root.putObject("document");
String filename = file.getOriginalFilename();
document.put("filename", filename);
document.put("extension", extensionOf(filename));
document.put("contentType", file.getContentType());
document.put("sizeBytes", content.length);
document.put("sha256", sha256(content));
// The bytes themselves, for steps that carry the document inside a JSON body
// (an attachment field, a signing payload) rather than as multipart.
document.put("base64", Base64.getEncoder().encodeToString(content));
if (looksLikePdf(content)) {
addPdfFacts(document, root, content, objectMapper);
}
ObjectNode run = root.putObject("run");
run.put("policyName", policyName);
run.put("runId", runId);
run.put("timestamp", Instant.now().toString());
return root;
}
/** PDF-only facts. A document we cannot parse still gets the basics above. */
private static void addPdfFacts(
ObjectNode document, ObjectNode root, byte[] content, ObjectMapper objectMapper) {
try (PDDocument pdf = Loader.loadPDF(content)) {
document.put("pageCount", pdf.getNumberOfPages());
document.put("encrypted", pdf.isEncrypted());
PDDocumentInformation info = pdf.getDocumentInformation();
document.put("title", info.getTitle());
document.put("author", info.getAuthor());
document.put("subject", info.getSubject());
document.put("keywords", info.getKeywords());
document.put("creator", info.getCreator());
document.put("producer", info.getProducer());
document.put("created", toIso(info.getCreationDate()));
document.put("modified", toIso(info.getModificationDate()));
addClassification(root, info, objectMapper);
addSensitivityLabel(root, pdf);
} catch (IOException | RuntimeException e) {
// An encrypted or malformed PDF is a normal thing to send to an external API; the
// extra facts are a convenience, not a precondition.
log.debug("Could not read PDF facts for the step context: {}", e.getMessage());
}
}
/** The classifier policy's verdict, so a call-out can act on it without re-classifying. */
private static void addClassification(
ObjectNode root, PDDocumentInformation info, ObjectMapper objectMapper) {
String raw = info.getCustomMetadataValue(PdfMetadataService.CLASSIFICATION_KEY);
if (raw == null || raw.isBlank()) {
return;
}
try {
JsonNode parsed = objectMapper.readTree(raw);
root.set("classification", parsed);
} catch (RuntimeException e) {
// Written by another tool; if it is not JSON, pass it through as text rather than drop
// it - the receiving system may still recognise it.
root.put("classification", raw);
}
}
/** The Purview label already on the document, if any. */
private static void addSensitivityLabel(ObjectNode root, PDDocument pdf) {
List<SensitivityLabel> labels = PdfSensitivityLabels.readAll(pdf);
if (labels.isEmpty()) {
return;
}
SensitivityLabel label = labels.get(0);
ObjectNode node = root.putObject("sensitivityLabel");
node.put("labelId", label.labelId());
node.put("name", label.name());
node.put("siteId", label.siteId());
node.put("method", label.method() == null ? null : label.method().name());
node.put("protected", label.isProtected());
}
/** Cheap check so a non-PDF never pays for a parse attempt. */
private static boolean looksLikePdf(byte[] content) {
return content.length > 4
&& content[0] == '%'
&& content[1] == 'P'
&& content[2] == 'D'
&& content[3] == 'F';
}
/**
* A content hash is the field external systems most often key on - dedupe, chain-of-custody,
* "have I already scanned this" - and they cannot compute it without the bytes we are sending.
*/
private static String sha256(byte[] content) {
try {
return HexFormat.of().formatHex(MessageDigest.getInstance("SHA-256").digest(content));
} catch (NoSuchAlgorithmException e) {
throw new IllegalStateException("SHA-256 is required by the Java platform", e);
}
}
private static String toIso(Calendar calendar) {
return calendar == null ? null : calendar.toInstant().toString();
}
private static String extensionOf(String filename) {
if (filename == null) {
return null;
}
int dot = filename.lastIndexOf('.');
return dot < 0 || dot == filename.length() - 1
? null
: filename.substring(dot + 1).toLowerCase(Locale.ROOT);
}
}
app/proprietary/src/main/java/stirling/software/proprietary/integration/api/ExternalApiCallController.java
+552
View File
@@ -0,0 +1,552 @@
package stirling.software.proprietary.integration.api;
import java.io.IOException;
import java.util.Base64;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import org.springframework.core.io.ByteArrayResource;
import org.springframework.core.io.Resource;
import org.springframework.http.HttpHeaders;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestHeader;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.multipart.MultipartFile;
import io.github.pixee.security.Filenames;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.tags.Tag;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.service.AutomationRunContext;
import stirling.software.common.service.InternalApiClient;
import stirling.software.common.util.TempFileManager;
import stirling.software.proprietary.service.AiToolResponseHeaders;
import tools.jackson.core.type.TypeReference;
import tools.jackson.databind.JsonNode;
import tools.jackson.databind.ObjectMapper;
import tools.jackson.databind.node.ObjectNode;
/**
* Posts the document flowing through a policy to a third-party HTTP API and folds the answer back
* into the run.
*
* <p>This is the generic integration step: rather than a bespoke connector per vendor, an operator
* defines an {@code API} connection (base URL + credentials) once and any policy can call a path
* under it. The connection owns the host and the credentials; the step owns only the path and the
* form fields, so a policy author can never aim the call somewhere else or read the secret.
*
* <p>Response handling is explicit rather than inferred, because the two useful behaviours destroy
* different things when guessed wrong:
*
* <ul>
* <li>{@code report} (default) - the document continues untouched and the API's answer rides
* along in {@link AiToolResponseHeaders#TOOL_REPORT}. For call-outs that inspect or notify. A
* {@code requireTrue} field turns the answer into a gate: the named JSON verdict must be true
* or the step fails, so a scanner's "not clean" actually stops the run.
* <li>{@code replace} - the response body <em>becomes</em> the document. For call-outs that
* transform. Fails loudly if the API returns JSON or an empty body, instead of silently
* dropping the document from the pipeline.
* </ul>
*/
@Slf4j
@RestController
@RequestMapping("/api/v1/integration")
@RequiredArgsConstructor
@Tag(name = "Integrations", description = "Third-party integration steps.")
public class ExternalApiCallController {
static final String MODE_REPORT = "report";
static final String MODE_REPLACE = "replace";
/**
* The report travels as an HTTP header, and Jetty caps a response header at 8KB by default. A
* body larger than this is summarised rather than risking a header the container refuses to
* write - which would fail the whole step over a merely verbose API.
*/
static final int MAX_REPORT_BODY_CHARS = 4096;
static final String BODY_MULTIPART = "multipart";
static final String BODY_JSON = "json";
static final String BODY_BINARY = "binary";
/** Field (multipart) and property (json) the auto-populated context is offered under. */
static final String CONTEXT_FIELD = "stirlingContext";
private final ApiConnectionResolver connectionResolver;
private final ExternalApiCaller caller;
private final ObjectMapper objectMapper;
private final TempFileManager tempFileManager;
private final ApplicationProperties applicationProperties;
@PostMapping(value = "/external-api-call", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
@Operation(
summary = "Send the document to an external API",
description =
"Sends the document to a path under a stored API connection's base URL and"
+ " either records the response as a step report or replaces the"
+ " document with it. Fields, path and headers may reference"
+ " {{document.*}}, {{classification.*}}, {{sensitivityLabel.*}} and"
+ " {{run.*}}. Type:SISO")
public ResponseEntity<Resource> call(
@RequestParam("fileInput") MultipartFile fileInput,
@RequestParam("connectionId") String connectionId,
@RequestParam(value = "path", required = false) String path,
@RequestParam(value = "method", defaultValue = "POST") String method,
@RequestParam(value = "bodyMode", defaultValue = BODY_MULTIPART) String bodyMode,
@RequestParam(value = "fileFieldName", defaultValue = "file") String fileFieldName,
@RequestParam(value = "responseMode", defaultValue = MODE_REPORT) String responseMode,
@RequestParam(value = "resultUrlPath", required = false) String resultUrlPath,
@RequestParam(value = "resultUrlHeader", required = false) String resultUrlHeader,
@RequestParam(value = "responseSelect", required = false) String responseSelect,
@RequestParam(value = "requireTrue", required = false) String requireTrue,
@RequestParam(value = "fields", required = false) String fields,
@RequestParam(value = "bodyTemplate", required = false) String bodyTemplate,
@RequestParam(value = "headers", required = false) String headers,
@RequestParam(value = "includeContext", defaultValue = "false") boolean includeContext,
@RequestParam(value = "includeFile", defaultValue = "true") boolean includeFile,
@RequestHeader(value = InternalApiClient.POLICY_NAME_HEADER, required = false)
String policyName,
@RequestHeader(value = AutomationRunContext.RUN_ID_HEADER, required = false)
String runId)
throws IOException {
String mode = normalise(responseMode, MODE_REPORT, MODE_REPORT, MODE_REPLACE);
String body = normalise(bodyMode, BODY_MULTIPART, BODY_MULTIPART, BODY_JSON, BODY_BINARY);
String verb = parseMethod(method);
Long id = ApiConnectionResolver.connectionId(connectionId);
if (id == null) {
throw new IllegalArgumentException("'connectionId' is required");
}
ApiConnectionSettings settings = connectionResolver.resolve(id);
String filename = safeFileName(fileInput.getOriginalFilename());
String contentType =
fileInput.getContentType() == null
? MediaType.APPLICATION_OCTET_STREAM_VALUE
: fileInput.getContentType();
byte[] content = fileInput.getBytes();
ObjectNode context =
DocumentContext.build(fileInput, content, policyName, runId, objectMapper);
ExternalApiCaller.Response response =
caller.dispatch(
settings,
verb,
Placeholders.resolve(path, context, Placeholders.Escaping.URL_PATH),
buildBody(
body,
bodyTemplate,
includeFile,
includeContext,
context,
fileFieldName,
filename,
contentType,
content,
resolveAll(parseJsonObject(fields, "fields"), context)),
validatedHeaders(resolveAll(parseJsonObject(headers, "headers"), context)));
if (!response.isSuccess()) {
// Fail the step: a policy that silently continued past a rejected call-out would
// deliver documents the external system believes it never approved.
throw new IOException(
"External API returned HTTP " + response.status() + summarise(response));
}
enforceVerdict(response, requireTrue);
return MODE_REPLACE.equals(mode)
? replaceDocument(
settings,
response,
filename,
resultUrlPath,
resultUrlHeader,
responseSelect)
: reportOnly(fileInput, filename, contentType, response);
}
/**
* Assemble the outbound body.
*
* <ul>
* <li>{@code multipart} - the file plus form fields, what most upload APIs expect.
* <li>{@code json} - a JSON object of the fields, with the context merged in and the file
* base64'd under {@code content}. For APIs that take a document as JSON, and for
* notify-style call-outs (with {@code includeFile=false}) that want the facts only.
* <li>{@code binary} - the raw bytes as the body. For APIs that want the file and nothing
* else; fields would have nowhere to go, so they are refused rather than dropped.
* </ul>
*/
private ExternalApiCaller.Body buildBody(
String bodyMode,
String bodyTemplate,
boolean includeFile,
boolean includeContext,
ObjectNode context,
String fileFieldName,
String filename,
String contentType,
byte[] content,
Map<String, String> fields)
throws IOException {
if (bodyTemplate != null && !bodyTemplate.isBlank()) {
return templatedBody(bodyTemplate, context, filename, contentType, content);
}
switch (bodyMode) {
case BODY_BINARY -> {
if (!fields.isEmpty()) {
throw new IllegalArgumentException(
"bodyMode 'binary' sends only the document, so 'fields' cannot be"
+ " sent; use 'headers' instead, or bodyMode 'multipart'.");
}
if (!includeFile) {
throw new IllegalArgumentException(
"bodyMode 'binary' with includeFile=false would send an empty body");
}
return ExternalApiCaller.raw(contentType, content);
}
case BODY_JSON -> {
ObjectNode json = objectMapper.createObjectNode();
fields.forEach(json::put);
if (includeContext) {
json.setAll(context);
}
if (includeFile) {
json.put("filename", filename);
json.put("contentType", contentType);
json.put("content", Base64.getEncoder().encodeToString(content));
}
return ExternalApiCaller.raw(
MediaType.APPLICATION_JSON_VALUE, objectMapper.writeValueAsBytes(json));
}
default -> {
Map<String, String> all = new LinkedHashMap<>(fields);
if (includeContext) {
all.put(CONTEXT_FIELD, objectMapper.writeValueAsString(context));
}
if (!includeFile) {
// Fields-only multipart: a notify-style call-out that wants the facts, not
// the document.
MultipartBody body = new MultipartBody();
body.addFields(all);
return new ExternalApiCaller.Body(body.contentType(), body.build());
}
return ExternalApiCaller.multipart(
fileFieldName, filename, contentType, content, all);
}
}
}
/**
* A caller-shaped JSON body: the template is resolved against the context, so an arbitrary
* vendor payload can be expressed as config. {@code {{document.base64}}} carries the file
* itself, which is how APIs that take a document nested inside a JSON document are reached.
*
* <p>The base64 is added to a copy of the context rather than the context proper: it is the
* size of the file, and {@code stirlingContext} must not silently grow by a whole document.
*/
private ExternalApiCaller.Body templatedBody(
String bodyTemplate,
ObjectNode context,
String filename,
String contentType,
byte[] content)
throws IOException {
JsonNode template;
try {
template = objectMapper.readTree(bodyTemplate);
} catch (Exception e) {
throw new IllegalArgumentException("api step 'bodyTemplate' must be valid JSON", e);
}
ObjectNode withFile = context.deepCopy();
ObjectNode document = (ObjectNode) withFile.get("document");
if (document != null) {
document.put("base64", Base64.getEncoder().encodeToString(content));
document.put("safeFilename", filename);
document.put("resolvedContentType", contentType);
}
JsonNode resolved = Placeholders.resolveTree(template, withFile);
return ExternalApiCaller.raw(
MediaType.APPLICATION_JSON_VALUE, objectMapper.writeValueAsBytes(resolved));
}
/** Resolve every value's placeholders against the context. */
private Map<String, String> resolveAll(Map<String, String> values, ObjectNode context) {
Map<String, String> out = new LinkedHashMap<>();
values.forEach(
(key, value) ->
out.put(
key,
Placeholders.resolve(value, context, Placeholders.Escaping.NONE)));
return out;
}
/** Per-step headers, held to the same rules as a connection's static headers. */
private Map<String, String> validatedHeaders(Map<String, String> headers) {
headers.forEach(
(name, value) -> {
if (!ExternalApiHeaders.isValidName(name)) {
throw new IllegalArgumentException(
"api step 'headers' has an invalid header name: " + name);
}
if (ExternalApiHeaders.isReserved(name)) {
throw new IllegalArgumentException(
"api step 'headers' must not set '"
+ name
+ "'; it is set by the connection or the client");
}
if (!ExternalApiHeaders.isValidValue(value)) {
// A resolved placeholder could carry a newline out of document metadata.
throw new IllegalArgumentException(
"api step 'headers' has an invalid value for '" + name + "'");
}
});
return headers;
}
private static String parseMethod(String method) {
String verb = method == null ? "POST" : method.trim().toUpperCase(Locale.ROOT);
// Only the verbs that carry a body; GET/DELETE would silently drop the document.
if (!List.of("POST", "PUT", "PATCH").contains(verb)) {
throw new IllegalArgumentException(
"'method' must be POST, PUT or PATCH; got " + method);
}
return verb;
}
private static String normalise(String value, String fallback, String... allowed) {
String out =
value == null || value.isBlank() ? fallback : value.trim().toLowerCase(Locale.ROOT);
if (!List.of(allowed).contains(out)) {
throw new IllegalArgumentException(
"must be one of " + String.join(", ", allowed) + "; got " + value);
}
return out;
}
/**
* Turn the response into the document that continues down the pipeline.
*
* <p>Three shapes of answer are accepted, because real APIs use all three: the document inline,
* a URL to fetch it from, or an archive to pick it out of. Anything else fails the step rather
* than putting a non-document into the pipeline for a later step to trip over.
*/
private ResponseEntity<Resource> replaceDocument(
ApiConnectionSettings settings,
ExternalApiCaller.Response response,
String requestFilename,
String resultUrlPath,
String resultUrlHeader,
String responseSelect)
throws IOException {
ExternalApiCaller.Response payload = response;
boolean followed = false;
String url = resultUrl(response, resultUrlPath, resultUrlHeader);
if (url != null) {
// The URL came out of the response, so ResultUrls decides whether it may be fetched.
payload =
caller.getResult(
settings, ResultUrls.validate(settings, url, applicationProperties));
followed = true;
if (!payload.isSuccess()) {
throw new IOException(
"Fetching the API's result URL returned HTTP " + payload.status());
}
}
if (payload.body().length == 0) {
throw new IOException(
"External API returned an empty body, so there is no document to replace with;"
+ " use responseMode=report to keep the original.");
}
if (payload.isJson() && !followed) {
throw new IOException(
"External API returned JSON, which cannot replace the document. Use"
+ " responseMode=report to keep the original and record the answer, or"
+ " set resultUrlPath if the JSON points at the document.");
}
String filename = ResultFiles.nameFor(payload, requestFilename);
Resource result = ResultFiles.asResource(payload.body(), filename);
if (ResultFiles.isArchive(result)) {
if (responseSelect == null || responseSelect.isBlank()) {
// Handing a .zip to a step that expects a PDF fails later and more obscurely.
throw new IOException(
"External API returned an archive; set 'responseSelect' (e.g. *.pdf, or an"
+ " index) to say which entry becomes the document.");
}
result = ResultFiles.selectFromArchive(result, responseSelect, tempFileManager);
filename = result.getFilename();
} else if (responseSelect != null && !responseSelect.isBlank()) {
throw new IOException(
"'responseSelect' was set but the API returned a single file, not an archive");
}
MediaType type =
payload.contentType() == null || ResultFiles.isArchiveName(filename)
? MediaType.APPLICATION_OCTET_STREAM
: MediaType.parseMediaType(payload.contentType().split(";")[0].trim());
return ResponseEntity.ok()
.contentType(type)
.header(
HttpHeaders.CONTENT_DISPOSITION,
"attachment; filename=\"" + filename + "\"")
.body(result);
}
/** The result URL the API pointed at, from the body or a header; null when neither is set. */
private String resultUrl(
ExternalApiCaller.Response response, String resultUrlPath, String resultUrlHeader) {
if (resultUrlHeader != null && !resultUrlHeader.isBlank()) {
String value = response.header(resultUrlHeader.trim());
if (value == null || value.isBlank()) {
throw new IllegalArgumentException(
"'resultUrlHeader' names '"
+ resultUrlHeader
+ "' but the response had no such header");
}
return value;
}
if (resultUrlPath == null || resultUrlPath.isBlank()) {
return null;
}
JsonNode node = response.bodyAsJson(objectMapper);
for (String segment : resultUrlPath.trim().split("\\.")) {
if (node == null) {
break;
}
node = node.get(segment);
}
if (node == null || !node.isValueNode() || node.asString().isBlank()) {
throw new IllegalArgumentException(
"'resultUrlPath' found no URL at '" + resultUrlPath + "' in the response");
}
return node.asString();
}
/**
* Gate the run on a boolean verdict in the API's JSON answer (e.g. Cloudmersive's {@code
* CleanResult}). When {@code requireTrue} names a field - dotted for a nested one - that field
* must be JSON {@code true}, or the step fails so the document is parked rather than delivered.
* Fail-closed: a missing field, a non-boolean, a false, or a non-JSON body all stop the run.
* This is what makes a scanner's "not clean" actually stop the pipeline.
*/
private void enforceVerdict(ExternalApiCaller.Response response, String requireTrue)
throws IOException {
if (requireTrue == null || requireTrue.isBlank()) {
return;
}
JsonNode node = response.isJson() ? response.bodyAsJson(objectMapper) : null;
for (String segment : requireTrue.trim().split("\\.")) {
if (node == null) {
break;
}
node = node.get(segment);
}
if (node == null || !node.asBoolean(false)) {
throw new IOException(
"External API verdict '"
+ requireTrue.trim()
+ "' was not true"
+ summarise(response)
+ "; the document was not approved, so the run was stopped.");
}
}
/** The document passes through; the API's answer rides in the report header. */
private ResponseEntity<Resource> reportOnly(
MultipartFile fileInput,
String filename,
String contentType,
ExternalApiCaller.Response response)
throws IOException {
return ResponseEntity.ok()
.contentType(MediaType.parseMediaType(contentType))
.header(
HttpHeaders.CONTENT_DISPOSITION,
"attachment; filename=\"" + filename + "\"")
.header(AiToolResponseHeaders.TOOL_REPORT, buildReport(response))
.body(new ByteArrayResource(fileInput.getBytes()));
}
/** A JSON object describing the call, small enough to survive as a header. */
private String buildReport(ExternalApiCaller.Response response) {
ObjectNode report = objectMapper.createObjectNode();
report.put("status", response.status());
report.put("contentType", response.contentType());
if (response.isJson()) {
try {
JsonNode parsed = objectMapper.readTree(response.bodyAsText());
String rendered = objectMapper.writeValueAsString(parsed);
if (rendered.length() <= MAX_REPORT_BODY_CHARS) {
report.set("body", parsed);
} else {
report.put("bodyTruncated", true);
report.put("body", rendered.substring(0, MAX_REPORT_BODY_CHARS));
}
} catch (Exception e) {
// Content-Type said JSON but the body is not; keep the step alive and say so.
report.put("bodyParseError", e.getMessage());
report.put("body", truncate(response.bodyAsText()));
}
} else {
report.put("bodyBytes", response.body().length);
}
return objectMapper.writeValueAsString(report);
}
/** A JSON object of string values, e.g. {@code {"policy":"strict"}}. */
private Map<String, String> parseJsonObject(String json, String what) {
if (json == null || json.isBlank()) {
return Map.of();
}
Map<String, Object> raw;
try {
raw =
objectMapper.readValue(
json, new TypeReference<LinkedHashMap<String, Object>>() {});
} catch (Exception e) {
throw new IllegalArgumentException("api step '" + what + "' must be a JSON object", e);
}
Map<String, String> out = new LinkedHashMap<>();
raw.forEach((key, value) -> out.put(key, value == null ? "" : value.toString()));
return out;
}
private String summarise(ExternalApiCaller.Response response) {
String text = truncate(response.bodyAsText());
return text.isBlank() ? "" : ": " + text;
}
private static String truncate(String text) {
if (text == null) {
return "";
}
String oneLine = text.replaceAll("\\s+", " ").trim();
return oneLine.length() <= MAX_REPORT_BODY_CHARS
? oneLine
: oneLine.substring(0, MAX_REPORT_BODY_CHARS) + "…";
}
private static String safeFileName(String originalFilename) {
String name = Filenames.toSimpleFileName(originalFilename);
return (name == null || name.isBlank()) ? "document" : name;
}
}
app/proprietary/src/main/java/stirling/software/proprietary/integration/api/ExternalApiCaller.java
+299
View File
@@ -0,0 +1,299 @@
package stirling.software.proprietary.integration.api;
import java.io.IOException;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.nio.charset.StandardCharsets;
import java.time.Duration;
import java.util.Base64;
import java.util.LinkedHashMap;
import java.util.Map;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.model.ApplicationProperties;
import tools.jackson.databind.JsonNode;
import tools.jackson.databind.ObjectMapper;
/**
* Performs the outbound call for an {@code API} connection.
*
* <p>Follows the established self-hosted outbound pattern (JDK {@link HttpClient}; see {@code
* AccountLinkClient}): the client is injectable so tests can drive a real local server without
* reaching the network.
*/
@Slf4j
@Service
public class ExternalApiCaller {
/**
* Cap on a response we will read into memory. An external API returning something enormous is a
* misconfiguration, and without a cap it would be a trivial way to OOM the server.
*/
static final int MAX_RESPONSE_BYTES = 64 * 1024 * 1024;
private static final Duration CONNECT_TIMEOUT = Duration.ofSeconds(10);
private final HttpClient httpClient;
private final ApplicationProperties applicationProperties;
private final ApiTokenCache tokenCache;
@Autowired
public ExternalApiCaller(
ApplicationProperties applicationProperties, ObjectMapper objectMapper) {
this(
HttpClient.newBuilder()
.connectTimeout(CONNECT_TIMEOUT)
// Following a redirect would re-target the request at a host the base URL
// never authorised, undoing ExternalApiPaths. Let the caller see the 3xx.
.followRedirects(HttpClient.Redirect.NEVER)
.build(),
applicationProperties,
objectMapper);
}
ExternalApiCaller(
HttpClient httpClient,
ApplicationProperties applicationProperties,
ObjectMapper objectMapper) {
this.httpClient = httpClient;
this.applicationProperties = applicationProperties;
this.tokenCache = new ApiTokenCache(httpClient, objectMapper);
}
/** What the external API sent back, before the step decides what to do with it. */
public record Response(
int status, String contentType, byte[] body, Map<String, String> headers) {
public Response {
headers = headers == null ? Map.of() : Map.copyOf(headers);
}
/** A response header by name, case-insensitively; null when absent. */
public String header(String name) {
for (Map.Entry<String, String> entry : headers.entrySet()) {
if (entry.getKey().equalsIgnoreCase(name)) {
return entry.getValue();
}
}
return null;
}
JsonNode bodyAsJson(ObjectMapper objectMapper) {
try {
return objectMapper.readTree(bodyAsText());
} catch (RuntimeException e) {
return null;
}
}
public boolean isSuccess() {
return status >= 200 && status < 300;
}
public boolean isJson() {
return contentType != null && contentType.toLowerCase().contains("json");
}
public String bodyAsText() {
return new String(body, StandardCharsets.UTF_8);
}
}
/**
* POST a document to {@code path} under the connection's base URL as multipart/form-data.
*
* @throws IOException on transport failure or an oversized response
*/
public Response postFile(
ApiConnectionSettings settings,
String path,
String fileFieldName,
String filename,
String fileContentType,
byte[] content,
Map<String, String> fields)
throws IOException {
return dispatch(
settings,
"POST",
path,
multipart(fileFieldName, filename, fileContentType, content, fields),
Map.of());
}
/** A request body plus the Content-Type that describes it. */
record Body(String contentType, HttpRequest.BodyPublisher publisher) {}
static Body multipart(
String fileFieldName,
String filename,
String fileContentType,
byte[] content,
Map<String, String> fields)
throws IOException {
MultipartBody body = new MultipartBody();
body.addFields(fields);
body.addFile(fileFieldName, filename, fileContentType, content);
return new Body(body.contentType(), body.build());
}
/** A body of caller-built bytes, e.g. a JSON document or the raw file. */
static Body raw(String contentType, byte[] content) {
return new Body(contentType, HttpRequest.BodyPublishers.ofByteArray(content));
}
/**
* Send {@code body} to {@code path} under the connection's base URL.
*
* @param method POST, PUT or PATCH - the verbs that carry a body
* @param extraHeaders per-step headers, already validated by the caller
*/
public Response dispatch(
ApiConnectionSettings settings,
String method,
String path,
Body body,
Map<String, String> extraHeaders)
throws IOException {
URI target = ExternalApiPaths.resolve(settings.baseUri(), path);
// Re-check at dispatch: save-time validation cannot see a DNS record re-pointed at a
// private address afterwards.
ApiIntegrationValidator.requirePublicHost(
settings, applicationProperties, "API connection base URL");
Response response = attempt(settings, method, target, body, extraHeaders);
if (response.status() == 401 && settings.authType() == ApiAuthType.TOKEN_LOGIN) {
// The cached token was rejected - expired early, or revoked. One fresh login and
// one retry; if that also 401s the credentials are wrong and the step says so.
log.debug("[external-api] token rejected by {}; re-authenticating", target.getHost());
tokenCache.invalidate(settings);
response = attempt(settings, method, target, body, extraHeaders);
}
return response;
}
private Response attempt(
ApiConnectionSettings settings,
String method,
URI target,
Body body,
Map<String, String> extraHeaders)
throws IOException {
HttpRequest.Builder request =
HttpRequest.newBuilder(target)
.timeout(Duration.ofSeconds(settings.timeoutSeconds()))
.header("Content-Type", body.contentType())
.method(method, body.publisher());
applyHeaders(request, settings);
// Step headers last so a step can override a connection default, but never the auth
// header: ExternalApiHeaders rejects reserved names before we get here.
extraHeaders.forEach(request::header);
return send(httpClient, request.build(), target);
}
/**
* GET an absolute result URL the API pointed us at.
*
* <p>Takes a {@link URI} rather than a string so it cannot be called with something unchecked:
* the only way to obtain one is {@link ResultUrls#validate}, which is where the host allowlist
* lives. Credentials are deliberately not sent - the URL is usually a presigned link on another
* host, and forwarding the connection's token there would leak it to a third party.
*/
public Response getResult(ApiConnectionSettings settings, URI target) throws IOException {
HttpRequest request =
HttpRequest.newBuilder(target)
.timeout(Duration.ofSeconds(settings.timeoutSeconds()))
.GET()
.build();
return send(httpClient, request, target);
}
/** GET {@code path} under the connection's base URL. */
public Response get(ApiConnectionSettings settings, String path) throws IOException {
URI target = ExternalApiPaths.resolve(settings.baseUri(), path);
ApiIntegrationValidator.requirePublicHost(
settings, applicationProperties, "API connection base URL");
HttpRequest.Builder request =
HttpRequest.newBuilder(target)
.timeout(Duration.ofSeconds(settings.timeoutSeconds()))
.GET();
applyHeaders(request, settings);
return send(httpClient, request.build(), target);
}
static Response send(HttpClient httpClient, HttpRequest request, URI target)
throws IOException {
HttpResponse<byte[]> response;
try {
response = httpClient.send(request, HttpResponse.BodyHandlers.ofByteArray());
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
throw new IOException("Interrupted calling " + safeTarget(target), e);
} catch (IOException e) {
// The message can carry the host but never the credentials, which live in headers.
throw new IOException(
"Failed to call " + safeTarget(target) + ": " + e.getMessage(), e);
}
byte[] body = response.body() == null ? new byte[0] : response.body();
if (body.length > MAX_RESPONSE_BYTES) {
throw new IOException(
"Response from "
+ safeTarget(target)
+ " exceeds the "
+ MAX_RESPONSE_BYTES
+ " byte limit");
}
String contentType = response.headers().firstValue("content-type").orElse(null);
Map<String, String> headers = new LinkedHashMap<>();
response.headers()
.map()
.forEach((name, values) -> headers.put(name, String.join(", ", values)));
log.debug("[external-api] {} -> HTTP {}", safeTarget(target), response.statusCode());
return new Response(response.statusCode(), contentType, body, headers);
}
private void applyHeaders(HttpRequest.Builder request, ApiConnectionSettings settings)
throws IOException {
settings.headers().forEach(request::header);
switch (settings.authType()) {
case BEARER -> request.header("Authorization", "Bearer " + settings.token());
case HEADER ->
request.header(
settings.headerName(),
settings.headerPrefix() == null
? settings.token()
: settings.headerPrefix() + " " + settings.token());
case BASIC ->
request.header(
"Authorization",
"Basic "
+ Base64.getEncoder()
.encodeToString(
(settings.username()
+ ":"
+ settings.password())
.getBytes(StandardCharsets.UTF_8)));
case TOKEN_LOGIN -> {
Map.Entry<String, String> auth = tokenCache.authHeader(settings);
request.header(auth.getKey(), auth.getValue());
}
case NONE -> {
/* no credentials */
}
}
}
/** Scheme, host and path only: a query string could carry a token an operator put there. */
private static String safeTarget(URI target) {
return target.getScheme() + "://" + target.getAuthority() + target.getPath();
}
}
app/proprietary/src/main/java/stirling/software/proprietary/integration/api/ExternalApiHeaders.java
+72
View File
@@ -0,0 +1,72 @@
package stirling.software.proprietary.integration.api;
import java.util.Locale;
import java.util.Set;
/**
* Validation for operator-supplied HTTP header names and values.
*
* <p>Header values reach the wire verbatim, so a value carrying CR/LF could splice extra headers -
* or a whole second request - into the stream. Names and values are therefore checked against the
* RFC 7230 grammar rather than trusted.
*/
public final class ExternalApiHeaders {
/**
* Headers a connection may not set as a static header. Authentication has exactly one path
* ({@code authType} + {@code token}) so credentials cannot be smuggled in as a "static" header
* that bypasses the auth validation; the rest are framing headers owned by the HTTP client,
* where a caller-set value would contradict the body actually sent.
*/
private static final Set<String> RESERVED =
Set.of(
"authorization",
"proxy-authorization",
"host",
"content-length",
"transfer-encoding",
"connection",
"upgrade",
"expect");
private ExternalApiHeaders() {}
/** RFC 7230 {@code token}: the only characters legal in a header name. */
public static boolean isValidName(String name) {
if (name == null || name.isEmpty()) {
return false;
}
for (int i = 0; i < name.length(); i++) {
if (!isTokenChar(name.charAt(i))) {
return false;
}
}
return true;
}
/** Visible ASCII, space and horizontal tab. Excludes CR/LF and NUL, which would inject. */
public static boolean isValidValue(String value) {
if (value == null) {
return false;
}
for (int i = 0; i < value.length(); i++) {
char c = value.charAt(i);
boolean printable = c >= 0x20 && c <= 0x7E;
if (!printable && c != '\t') {
return false;
}
}
return true;
}
public static boolean isReserved(String name) {
return name != null && RESERVED.contains(name.toLowerCase(Locale.ROOT));
}
private static boolean isTokenChar(char c) {
return (c >= 'a' && c <= 'z')
|| (c >= 'A' && c <= 'Z')
|| (c >= '0' && c <= '9')
|| "!#$%&'*+-.^_`|~".indexOf(c) >= 0;
}
}
app/proprietary/src/main/java/stirling/software/proprietary/integration/api/ExternalApiPaths.java
+120
View File
@@ -0,0 +1,120 @@
package stirling.software.proprietary.integration.api;
import java.net.URI;
import java.net.URISyntaxException;
import java.util.Locale;
/**
* Resolves a step-supplied relative path under a connection's operator-set base URL.
*
* <p>This is the control that keeps the external-API step from being an SSRF primitive. The base
* URL comes from an {@code IntegrationConfig} only someone with manage rights can edit; the path
* comes from a pipeline step, which is a far weaker trust boundary. Everything here exists to
* guarantee that a path can address a resource <em>under</em> the base and nothing else.
*
* <p>{@link URI#resolve} is deliberately not used: resolving the protocol-relative {@code
* //evil.example} against {@code https://api.example.com/v1} yields {@code https://evil.example},
* silently changing host. Instead the path is screened, appended textually, normalised, and then
* the result is re-checked against the base - so a miss in the screen is still caught by the check.
*/
public final class ExternalApiPaths {
private ExternalApiPaths() {}
/**
* @param base the connection's base URL, already validated as http(s) with a host
* @param path a relative path, optionally with a query string; blank means the base itself
* @throws IllegalArgumentException if the path is absolute, escapes the base, or carries
* characters that could split the request line
*/
public static URI resolve(URI base, String path) {
if (path == null || path.isBlank()) {
return base;
}
String candidate = path.trim();
screen(candidate);
if (!candidate.startsWith("/")) {
candidate = "/" + candidate;
}
URI resolved;
try {
resolved = new URI(base + candidate).normalize();
} catch (URISyntaxException e) {
throw new IllegalArgumentException(
"api step 'path' is not a valid URL path: " + path, e);
}
requireSameOrigin(base, resolved, path);
requireUnderBasePath(base, resolved, path);
return resolved;
}
/** Reject the shapes that could retarget the request before it is even assembled. */
private static void screen(String path) {
if (path.contains("://") || path.startsWith("//")) {
throw new IllegalArgumentException(
"api step 'path' must be relative to the connection's base URL, not an"
+ " absolute or protocol-relative URL: "
+ path);
}
for (int i = 0; i < path.length(); i++) {
char c = path.charAt(i);
// Control characters and spaces can split the request line; a backslash is normalised
// to '/' by some servers and would sidestep the traversal check below.
if (c <= 0x20 || c == 0x7F || c == '\\') {
throw new IllegalArgumentException(
"api step 'path' contains an illegal character: " + path);
}
}
if (path.indexOf('#') >= 0) {
throw new IllegalArgumentException(
"api step 'path' must not contain a fragment: " + path);
}
// Percent-encoded dots would survive the normalise() below and be decoded by the target, so
// a traversal must not be smuggled past us in encoded form.
//
// Only dots are rejected. An encoded slash or backslash is legitimate: Placeholders encodes
// substituted values, so a filename containing '/' arrives here as %2F, where it is data
// inside one segment rather than structure. Rejecting those would refuse ordinary filenames
// while doing nothing for traversal, which needs the dots.
String lower = path.toLowerCase(Locale.ROOT);
if (lower.contains("%2e")) {
throw new IllegalArgumentException(
"api step 'path' must not percent-encode dots: " + path);
}
}
private static void requireSameOrigin(URI base, URI resolved, String original) {
boolean sameOrigin =
equalsIgnoreCase(base.getScheme(), resolved.getScheme())
&& equalsIgnoreCase(base.getHost(), resolved.getHost())
&& base.getPort() == resolved.getPort()
&& resolved.getUserInfo() == null;
if (!sameOrigin) {
throw new IllegalArgumentException(
"api step 'path' would change the target host; it must stay under the"
+ " connection's base URL: "
+ original);
}
}
private static void requireUnderBasePath(URI base, URI resolved, String original) {
String basePath = base.getPath() == null ? "" : base.getPath();
String resolvedPath = resolved.getPath() == null ? "" : resolved.getPath();
// The base URL has its trailing slash stripped at parse time, so a base path of "/v1"
// must match "/v1" exactly or be followed by a separator - never "/v1betray".
boolean under =
basePath.isEmpty()
|| resolvedPath.equals(basePath)
|| resolvedPath.startsWith(basePath + "/");
if (!under) {
throw new IllegalArgumentException(
"api step 'path' escapes the connection's base path: " + original);
}
}
private static boolean equalsIgnoreCase(String a, String b) {
return a == null ? b == null : a.equalsIgnoreCase(b);
}
}
app/proprietary/src/main/java/stirling/software/proprietary/integration/api/IntegrationStepValidator.java
+68
View File
@@ -0,0 +1,68 @@
package stirling.software.proprietary.integration.api;
import java.util.Map;
import org.springframework.stereotype.Component;
import lombok.RequiredArgsConstructor;
import stirling.software.proprietary.integration.model.IntegrationType;
import stirling.software.proprietary.policy.engine.PipelineStepValidator;
import stirling.software.proprietary.policy.model.PipelineStep;
/**
* Authorization-checks the {@code connectionId} of any integration step, on the request thread.
*
* <p>This is what stops an integration step being a confused deputy. A step names a connection by
* id, and the worker thread that runs it has no principal - so {@link ApiConnectionResolver} lets
* the lookup through unchecked there, exactly as the S3 resolver does. Without this validator a
* caller could put any id in a step and have the server dial that tenant's endpoint with that
* tenant's stored credentials. Resolving here, while the caller is still on the thread, forces the
* ownership check to run.
*
* <p>Registered as a {@link PipelineStepValidator} so both entry points cover it: save-time
* validation of a stored policy, and {@code PolicyController}'s ad-hoc gate.
*/
@Component
@RequiredArgsConstructor
public class IntegrationStepValidator implements PipelineStepValidator {
static final String CONNECTION_ID_PARAM = "connectionId";
private static final String INTEGRATION_PREFIX = "/api/v1/integration/";
/**
* Which connection type each integration step dereferences. A step under {@link
* #INTEGRATION_PREFIX} that is absent here is rejected rather than waved through, so a new
* endpoint cannot quietly skip this check by forgetting to register.
*/
private static final Map<String, IntegrationType> STEP_CONNECTION_TYPES =
Map.of(
"/api/v1/integration/external-api-call", IntegrationType.API,
"/api/v1/integration/purview-apply-label", IntegrationType.PURVIEW,
"/api/v1/integration/purview-read-label", IntegrationType.PURVIEW,
"/api/v1/integration/consigno-submit", IntegrationType.CONSIGNO,
"/api/v1/integration/consigno-fetch-signed", IntegrationType.CONSIGNO);
private final ApiConnectionResolver connectionResolver;
@Override
public void validate(PipelineStep step) {
String operation = step.operation();
if (operation == null || !operation.startsWith(INTEGRATION_PREFIX)) {
return;
}
IntegrationType type = STEP_CONNECTION_TYPES.get(operation);
if (type == null) {
throw new IllegalArgumentException("unknown integration step: " + operation);
}
Long connectionId =
ApiConnectionResolver.connectionId(step.parameters().get(CONNECTION_ID_PARAM));
if (connectionId == null) {
throw new IllegalArgumentException(
operation + " requires a '" + CONNECTION_ID_PARAM + "' parameter");
}
// Throws if the connection is missing, the wrong type, disabled, or not usable by the
// caller. The parsed settings are discarded: this call is the check.
connectionResolver.resolveConfig(connectionId, type);
}
}
app/proprietary/src/main/java/stirling/software/proprietary/integration/api/MultipartBody.java
+101
View File
@@ -0,0 +1,101 @@
package stirling.software.proprietary.integration.api;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.net.http.HttpRequest;
import java.nio.charset.StandardCharsets;
import java.security.SecureRandom;
import java.util.Base64;
import java.util.Map;
/**
* Builds a {@code multipart/form-data} body for the JDK HTTP client, which has no multipart
* publisher of its own.
*
* <p>The body is assembled in memory. Callers bound the document size before getting here; the
* external-API step is for API-shaped payloads, not bulk transfer.
*/
final class MultipartBody {
private final String boundary;
private final ByteArrayOutputStream out = new ByteArrayOutputStream();
MultipartBody() {
byte[] random = new byte[16];
new SecureRandom().nextBytes(random);
this.boundary =
"StirlingBoundary" + Base64.getUrlEncoder().withoutPadding().encodeToString(random);
}
String contentType() {
return "multipart/form-data; boundary=" + boundary;
}
/**
* @throws IllegalArgumentException if the <em>name</em> could break out of its part header;
* names come from step parameters, so they are checked rather than trusted
*/
MultipartBody addField(String name, String value) throws IOException {
requireSafe(name, "field name");
writeAscii("--" + boundary + "\r\n");
writeAscii("Content-Disposition: form-data; name=\"" + name + "\"\r\n\r\n");
// The value is body, not header: quotes, newlines and backslashes are ordinary data here
// and must survive untouched. Checking it like a header rejected every JSON value - which
// is most of them, the auto-populated context included.
out.write(value.getBytes(StandardCharsets.UTF_8));
writeAscii("\r\n");
return this;
}
MultipartBody addFile(String name, String filename, String contentType, byte[] content)
throws IOException {
requireSafe(name, "file field name");
requireSafe(filename, "filename");
writeAscii("--" + boundary + "\r\n");
writeAscii(
"Content-Disposition: form-data; name=\""
+ name
+ "\"; filename=\""
+ filename
+ "\"\r\n");
writeAscii("Content-Type: " + contentType + "\r\n\r\n");
out.write(content);
writeAscii("\r\n");
return this;
}
HttpRequest.BodyPublisher build() throws IOException {
writeAscii("--" + boundary + "--\r\n");
return HttpRequest.BodyPublishers.ofByteArray(out.toByteArray());
}
MultipartBody addFields(Map<String, String> fields) throws IOException {
for (Map.Entry<String, String> entry : fields.entrySet()) {
addField(entry.getKey(), entry.getValue());
}
return this;
}
/**
* A quote, CR, LF or backslash in a <em>part header</em> - a field name or filename - would let
* it close the quoted string and forge headers of its own. Values are not checked: they are
* body, and the boundary that delimits them is 16 random bytes minted per request, so a value
* cannot end its own part.
*/
private static void requireSafe(String value, String what) {
if (value == null) {
throw new IllegalArgumentException("api step " + what + " must not be null");
}
if (value.indexOf('"') >= 0
|| value.indexOf('\r') >= 0
|| value.indexOf('\n') >= 0
|| value.indexOf('\\') >= 0) {
throw new IllegalArgumentException(
"api step " + what + " contains an illegal character: " + value);
}
}
private void writeAscii(String text) throws IOException {
out.write(text.getBytes(StandardCharsets.UTF_8));
}
}
app/proprietary/src/main/java/stirling/software/proprietary/integration/api/Placeholders.java
+153
View File
@@ -0,0 +1,153 @@
package stirling.software.proprietary.integration.api;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import tools.jackson.databind.JsonNode;
import tools.jackson.databind.node.ArrayNode;
import tools.jackson.databind.node.ObjectNode;
import tools.jackson.databind.node.StringNode;
/**
* Substitutes {@code {{dotted.path}}} references against the {@link DocumentContext}.
*
* <p>This is what lets one step satisfy APIs that disagree about payload shape. Rather than a
* connector per vendor, an operator writes the field names the vendor expects and fills them from
* context - {@code {"sha256": "{{document.sha256}}", "class": "{{sensitivityLabel.name}}"}}.
*
* <p>Deliberately not a template language: dotted lookup and nothing else. No expressions, no
* control flow, no method calls - a step definition is lower-trust than server config, and the
* whole point of a template engine (evaluating what it is given) is the thing to avoid here.
*/
final class Placeholders {
private static final Pattern PLACEHOLDER = Pattern.compile("\\{\\{\\s*([\\w.]+)\\s*}}");
/** How a resolved value is escaped for the position it lands in. */
enum Escaping {
/** Verbatim: form fields and header values, which are validated separately. */
NONE,
/** Percent-encoded: a path segment, where a stray slash would change the target. */
URL_PATH
}
private Placeholders() {}
/**
* @param template text that may contain {@code {{...}}} references; null passes through
* @param context the object to resolve against
* @throws IllegalArgumentException if a reference names something the context does not hold, so
* a typo surfaces as an error instead of silently sending an empty value
*/
static String resolve(String template, JsonNode context, Escaping escaping) {
if (template == null || template.isEmpty()) {
return template;
}
Matcher matcher = PLACEHOLDER.matcher(template);
StringBuilder out = new StringBuilder();
while (matcher.find()) {
String path = matcher.group(1);
JsonNode value = lookup(context, path);
if (value == null || value.isMissingNode()) {
throw new IllegalArgumentException(
"unknown placeholder '{{"
+ path
+ "}}'; available: document.*, classification.*,"
+ " sensitivityLabel.*, run.*");
}
matcher.appendReplacement(out, Matcher.quoteReplacement(render(value, escaping)));
}
matcher.appendTail(out);
return out.toString();
}
/**
* Resolve every string in a JSON tree, in place, leaving structure and non-strings alone.
*
* <p>This is what lets one step post an arbitrary vendor-shaped body - a nested {@code
* documents[0].data} as readily as a flat field - without a connector per vendor.
*/
static JsonNode resolveTree(JsonNode node, JsonNode context) {
if (node instanceof ObjectNode object) {
for (String name : new java.util.ArrayList<>(object.propertyNames())) {
object.set(name, resolveTree(object.get(name), context));
}
return object;
}
if (node instanceof ArrayNode array) {
for (int i = 0; i < array.size(); i++) {
array.set(i, resolveTree(array.get(i), context));
}
return array;
}
if (node != null && node.isString()) {
return StringNode.valueOf(resolve(node.asString(), context, Escaping.NONE));
}
return node;
}
/** Whether the text references anything at all, so callers can skip resolving. */
static boolean hasPlaceholder(String text) {
return text != null && PLACEHOLDER.matcher(text).find();
}
private static JsonNode lookup(JsonNode context, String path) {
JsonNode node = context;
for (String segment : path.split("\\.")) {
if (node == null || !node.isObject()) {
return null;
}
node = node.get(segment);
}
return node;
}
/**
* A null in context renders empty rather than the literal "null": absent metadata is a normal
* state, and "null" in a vendor's field would be a value, not an absence.
*/
private static String render(JsonNode value, Escaping escaping) {
String text;
if (value.isNull()) {
text = "";
} else if (value.isValueNode()) {
text = value.asString();
} else {
// An object or array (e.g. {{classification}}) renders as its JSON.
text = value.toString();
}
return escaping == Escaping.URL_PATH ? urlEncodePathSegment(text) : text;
}
/**
* Encode for a path segment: a filename is the likeliest value to land in a path and may carry
* a slash, which would otherwise read as structure rather than data.
*
* <p>Dots are left alone even though a traversal is made of them. Encoding them would be worse:
* {@code %2E%2E} survives {@link java.net.URI#normalize()} and gets decoded by the target, so
* the traversal would arrive intact and unexamined. Left raw, {@code ..} normalises here and is
* caught by {@code ExternalApiPaths}' under-the-base check - the one place that can actually
* see it.
*/
private static String urlEncodePathSegment(String text) {
StringBuilder out = new StringBuilder(text.length());
for (byte b : text.getBytes(java.nio.charset.StandardCharsets.UTF_8)) {
char c = (char) (b & 0xFF);
// RFC 3986 unreserved.
boolean unreserved =
(c >= 'a' && c <= 'z')
|| (c >= 'A' && c <= 'Z')
|| (c >= '0' && c <= '9')
|| c == '-'
|| c == '.'
|| c == '_'
|| c == '~';
if (unreserved) {
out.append(c);
} else {
out.append('%').append(String.format("%02X", b & 0xFF));
}
}
return out.toString();
}
}
app/proprietary/src/main/java/stirling/software/proprietary/integration/api/ResultFiles.java
+215
View File
@@ -0,0 +1,215 @@
package stirling.software.proprietary.integration.api;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import org.springframework.core.io.ByteArrayResource;
import org.springframework.core.io.Resource;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.ZipExtractionUtils;
/**
* Works out which bytes, and under which name, a response should contribute to the pipeline.
*
* <p>Three things go wrong if this is left implicit:
*
* <ul>
* <li><b>The name.</b> A step that replaces the document must name it for what came back, not for
* what went out. Keeping the inbound name means a PDF-to-DOCX call-out yields a DOCX called
* {@code .pdf}, and the next step's type check either waves it through or rejects it for the
* wrong reason. The response's own {@code Content-Disposition} or {@code Content-Type} is the
* only honest source.
* <li><b>Archives.</b> Plenty of APIs answer with a ZIP even when one file was sent - ConsignO
* returns "PDF (single) or ZIP (multiple)". Handing a {@code .zip} to a step expecting a PDF
* is a confusing failure, so a step can select what it wanted out of the archive.
* <li><b>Nothing useful at all.</b> An empty body or an error page is not a document, and saying
* so beats letting it flow onward as one.
* </ul>
*/
final class ResultFiles {
/** Extensions we can name from a content type; anything else keeps the server's filename. */
private static final Map<String, String> EXTENSION_BY_TYPE =
Map.ofEntries(
Map.entry("application/pdf", "pdf"),
Map.entry("application/zip", "zip"),
Map.entry("application/json", "json"),
Map.entry("text/plain", "txt"),
Map.entry("text/html", "html"),
Map.entry("image/png", "png"),
Map.entry("image/jpeg", "jpg"),
Map.entry("image/tiff", "tiff"),
Map.entry("application/msword", "doc"),
Map.entry(
"application/vnd.openxmlformats-officedocument.wordprocessingml.document",
"docx"),
Map.entry("application/vnd.ms-excel", "xls"),
Map.entry(
"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
"xlsx"));
private ResultFiles() {}
/**
* The filename to give the returned bytes.
*
* <p>Prefers what the server said ({@code Content-Disposition}), then the base name of the
* request with an extension derived from {@code Content-Type}, and only then the original name
* unchanged.
*/
static String nameFor(ExternalApiCaller.Response response, String requestFilename) {
String disposition = response.header("content-disposition");
String fromServer = filenameFromDisposition(disposition);
if (fromServer != null) {
return fromServer;
}
String extension = extensionFor(response.contentType());
if (extension == null) {
return requestFilename;
}
return baseName(requestFilename) + "." + extension;
}
/**
* Pick the file a step asked for out of an archive.
*
* @param select a glob such as {@code *.pdf}, or a 0-based index such as {@code 1}
* @throws IOException if nothing in the archive matches, naming what was there - a silent pick
* of the wrong file would be worse than a failed step
*/
static Resource selectFromArchive(
Resource archive, String select, TempFileManager tempFileManager) throws IOException {
List<Resource> entries = ZipExtractionUtils.extractZip(archive, tempFileManager);
if (entries.isEmpty()) {
throw new IOException("The API returned an empty archive");
}
Integer index = asIndex(select);
if (index != null) {
if (index < 0 || index >= entries.size()) {
throw new IOException(
"'responseSelect' asked for entry "
+ index
+ " but the archive has "
+ entries.size()
+ ": "
+ names(entries));
}
return entries.get(index);
}
List<Resource> matches = new ArrayList<>();
for (Resource entry : entries) {
if (matchesGlob(entry.getFilename(), select)) {
matches.add(entry);
}
}
if (matches.isEmpty()) {
throw new IOException(
"'responseSelect' matched nothing in the archive; it holds " + names(entries));
}
if (matches.size() > 1) {
// Taking the first would be a coin toss the operator did not ask for.
throw new IOException(
"'responseSelect' matched "
+ matches.size()
+ " entries ("
+ names(matches)
+ "); narrow it, or use an index");
}
return matches.get(0);
}
/** Whether the chosen name is itself an archive, so its content type is not the entry's. */
static boolean isArchiveName(String filename) {
return filename != null && filename.toLowerCase(Locale.ROOT).endsWith(".zip");
}
static boolean isArchive(Resource resource) throws IOException {
return ZipExtractionUtils.isZip(resource);
}
static Resource asResource(byte[] content, String filename) {
return new ByteArrayResource(content) {
@Override
public String getFilename() {
return filename;
}
};
}
/** Only {@code *} is supported, and only against the entry's own name. */
private static boolean matchesGlob(String filename, String glob) {
if (filename == null) {
return false;
}
String name = filename.toLowerCase(Locale.ROOT);
String pattern = glob.trim().toLowerCase(Locale.ROOT);
String regex =
java.util.Arrays.stream(pattern.split("\\*", -1))
.map(java.util.regex.Pattern::quote)
.reduce((a, b) -> a + ".*" + b)
.orElse("");
return name.matches(regex);
}
private static Integer asIndex(String select) {
try {
return Integer.valueOf(select.trim());
} catch (NumberFormatException e) {
return null;
}
}
private static String names(List<Resource> entries) {
return entries.stream().map(Resource::getFilename).toList().toString();
}
/** {@code attachment; filename="signed.pdf"} or its RFC 5987 {@code filename*} form. */
private static String filenameFromDisposition(String disposition) {
if (disposition == null) {
return null;
}
for (String part : disposition.split(";")) {
String token = part.trim();
String value = null;
if (token.regionMatches(true, 0, "filename=", 0, 9)) {
value = token.substring(9).trim();
} else if (token.regionMatches(true, 0, "filename*=", 0, 10)) {
value = token.substring(10).trim();
int tick = value.lastIndexOf('\'');
if (tick >= 0) {
value = value.substring(tick + 1);
}
}
if (value == null) {
continue;
}
if (value.length() >= 2 && value.startsWith("\"") && value.endsWith("\"")) {
value = value.substring(1, value.length() - 1);
}
// The name comes from the remote server, so it is treated as data: strip any path it
// tries to bring with it rather than letting it steer where anything is written.
String simple = io.github.pixee.security.Filenames.toSimpleFileName(value);
if (simple != null && !simple.isBlank()) {
return simple;
}
}
return null;
}
private static String extensionFor(String contentType) {
if (contentType == null) {
return null;
}
String type = contentType.split(";")[0].trim().toLowerCase(Locale.ROOT);
return EXTENSION_BY_TYPE.get(type);
}
private static String baseName(String filename) {
int dot = filename.lastIndexOf('.');
return dot <= 0 ? filename : filename.substring(0, dot);
}
}

Some files were not shown because too many files have changed in this diff Show More