mirror of
https://github.com/Stirling-Tools/Stirling-PDF.git
synced 2026-09-03 05:10:16 +03:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
40ccbc15cc | ||
|
|
62944d7423 | ||
|
|
7ce98d29f2 | ||
|
|
65c01e8078 | ||
|
|
4700542c75 | ||
|
|
9d01866c83 | ||
|
|
b35329c8f5 | ||
|
|
4dc0927104 | ||
|
|
b4a264239c | ||
|
|
bbd4d2c3ac | ||
|
|
4d207f0c3f | ||
|
|
8a5470dd01 | ||
|
|
999b5e5995 | ||
|
|
66b80a80c0 | ||
|
|
e05b7f12da | ||
|
|
dff3101ca7 | ||
|
|
af1acb68d5 | ||
|
|
22ec0947c9 | ||
|
|
ba404d3f90 | ||
|
|
a380a82234 | ||
|
|
3813ca360e | ||
|
|
1681b5d298 | ||
|
|
831bd4fe94 | ||
|
|
4fbb2fe885 | ||
|
|
54bf32485f | ||
|
|
a1b1f974a0 | ||
|
|
29002d0b82 | ||
|
|
1e2895a79f | ||
|
|
67e10138b5 | ||
|
|
8de94ff152 | ||
|
|
b3875d3149 | ||
|
|
357eb77f94 | ||
|
|
718277a934 | ||
|
|
3bf0019d7c | ||
|
|
7e76097ac1 | ||
|
|
ce7f74a3c1 | ||
|
|
50c0f2bcb5 | ||
|
|
fab00f3fe2 | ||
|
|
b5b9cc443f | ||
|
|
a4ac034a18 | ||
|
|
e24a30828b | ||
|
|
e2ea720fc8 | ||
|
|
a4ffdc7831 | ||
|
|
621731bda1 | ||
|
|
5e3e89ccb2 | ||
|
|
f60a75c253 | ||
|
|
401894a065 | ||
|
|
15423c6479 | ||
|
|
f4560cccb9 | ||
|
|
43a6ff3066 | ||
|
|
14def378bf | ||
|
|
49cc3f88f9 | ||
|
|
79dc7d5615 | ||
|
|
bda9cebc5c | ||
|
|
0fc2958daa | ||
|
|
c19d56de22 | ||
|
|
1954d20910 | ||
|
|
b6b5077467 | ||
|
|
a63ab2db7b | ||
|
|
1ba9092999 | ||
|
|
99887f77a6 | ||
|
|
1b3e757686 | ||
|
|
bb830e5711 | ||
|
|
f4d760b139 | ||
|
|
4cfe29139a | ||
|
|
0be0c0839b | ||
|
|
c6e84a2124 | ||
|
|
72e8ba2951 | ||
|
|
4230d2902b | ||
|
|
2c1a666889 | ||
|
|
47fe4a1d06 | ||
|
|
b509fae0c8 | ||
|
|
8b179fbc55 | ||
|
|
ecddce9944 | ||
|
|
398f473316 | ||
|
|
bc96eed040 | ||
|
|
d271b8f357 | ||
|
|
6b2ab5a743 | ||
|
|
275d70242e | ||
|
|
1835e6fe6b | ||
|
|
939afef14f | ||
|
|
2118de0bb7 | ||
|
|
963fe6c0cf |
@@ -0,0 +1,137 @@
|
||||
---
|
||||
name: pr-quiz
|
||||
description: >-
|
||||
Quiz the PR author on their own branch before they request review, to prove they
|
||||
actually understand the change - especially code an AI wrote for them. Scopes the
|
||||
branch diff vs its base, reads the changed code, then asks graded questions about
|
||||
what changed, why, how it works, what it could break, and which edge cases it must
|
||||
handle. Presents all questions first, waits for the author's answers, then grades
|
||||
each honestly against the real code (Correct / Partial / Incorrect with the true
|
||||
answer and file:line), scores it, and gives a readiness verdict that names the
|
||||
areas to re-study before asking humans to review. Use when asked to quiz me on my
|
||||
PR/branch, "test my understanding before review", a self-check gate before opening
|
||||
a PR, or before requesting reviewers. Administered as an interactive
|
||||
multiple-choice quiz (clickable options) by default; pass --free-text for
|
||||
written answers, --questions N to set count, --save to write a scorecard.
|
||||
argument-hint: "[branch-or-base-ref] [--questions N] [--free-text] [--save]"
|
||||
allowed-tools: Bash, Read, Grep, Glob, Write, AskUserQuestion
|
||||
---
|
||||
|
||||
# PR Quiz
|
||||
|
||||
Test whether the **author** genuinely understands their own branch before they ask
|
||||
other people to spend time reviewing it. This is a self-check gate: the point is to
|
||||
catch changes - often AI-written - that the author would not be able to explain or
|
||||
defend in review. Be a fair but honest examiner, not a pushover.
|
||||
|
||||
`$ARGUMENTS` may name a base ref or branch to diff against; default is this branch
|
||||
vs where it forked from the main line. Flags:
|
||||
- `--questions N` - target N questions (else scale to diff size, see below).
|
||||
- `--free-text` - administer as a written numbered list instead of the default
|
||||
interactive multiple-choice.
|
||||
- `--save` - also write a scorecard file after grading.
|
||||
|
||||
## Integrity rules (read first - the whole skill depends on these)
|
||||
|
||||
1. **Present every question before revealing any answer.** Ask, then wait. Never
|
||||
show the answer key alongside the questions.
|
||||
2. **Do not give hints or the answer while the quiz is open.** If the author asks
|
||||
"what's the answer?" or "is it X?" before committing, decline warmly and tell
|
||||
them to give their best answer first - guessing is part of the signal.
|
||||
3. **Grade truthfully.** Vague, hand-wavy, or "the AI did it" non-answers are
|
||||
Partial or Incorrect, not Correct. Do not inflate the score to be nice; a false
|
||||
pass defeats the entire purpose.
|
||||
4. **Ground everything in code you actually read.** Every question and every model
|
||||
answer must trace to a real line in the diff. Cite `path:line`. No trivia
|
||||
("how many lines?"), no invented behavior.
|
||||
5. **Credit real understanding.** If the author explains it correctly in their own
|
||||
words, mark it Correct even if worded differently than your key.
|
||||
|
||||
## Process
|
||||
|
||||
### 1. Scope the change (silently)
|
||||
- Find the base. Prefer the fork point off the main line so the quiz covers only
|
||||
this branch's work:
|
||||
```bash
|
||||
git fetch -q origin 2>/dev/null; \
|
||||
BASE=$(git merge-base HEAD origin/main 2>/dev/null || git merge-base HEAD main); \
|
||||
git diff --stat "$BASE"...HEAD
|
||||
```
|
||||
If `$ARGUMENTS` names a ref, diff against that instead.
|
||||
- If the diff is empty, stop and say there's nothing to quiz on.
|
||||
- Read commit messages / PR description for the *stated* intent, but verify it
|
||||
against the actual diff - a mismatch is itself a good question.
|
||||
|
||||
### 2. Understand the code well enough to examine on it
|
||||
Read the full diff plus enough surrounding context and related files to answer
|
||||
every question you plan to ask. You cannot grade understanding you don't have.
|
||||
Note the non-obvious parts: the design decisions, the risky lines, the edge cases,
|
||||
the cross-file ripples, and anything that violates or upholds repo conventions
|
||||
(for this repo e.g. `@app/*` import layering, all file ops via FileContext,
|
||||
Jackson 3 / Spring Boot 4 APIs, engine typed-contract boundaries).
|
||||
|
||||
### 3. Build the question set
|
||||
Scale count to the change unless `--questions N` is given:
|
||||
small (< ~50 changed lines) 3-4, medium 5-8, large 9-12. Cap at 12.
|
||||
Draw from these categories - weight toward the ones the diff actually exercises:
|
||||
- **Intent** - what problem this solves; why it was needed now.
|
||||
- **Mechanism** - how a specific non-trivial piece actually works ("walk me
|
||||
through what `foo()` does when called with X").
|
||||
- **Decisions & alternatives** - why this approach over an obvious alternative;
|
||||
what a reviewer would reasonably push back on.
|
||||
- **Blast radius** - what else this touches or could break; what you'd retest.
|
||||
- **Edge cases** - inputs/states the change must handle (null, empty, large,
|
||||
concurrent, error paths).
|
||||
- **Conventions & correctness** - does it follow the repo's rules; is there a
|
||||
latent bug the author should be able to spot.
|
||||
Prefer questions the author can only answer if they read and understood the code.
|
||||
Keep a private answer key with `path:line` for each - do **not** show it yet.
|
||||
|
||||
### 4. Administer the quiz
|
||||
- **Default (multiple choice):** use the `AskUserQuestion` tool. Per question write
|
||||
3-4 options where **every** option is independently plausible - each distractor a
|
||||
real-but-wrong reading of the code, not filler. Two hard rules so the answer
|
||||
can't be spotted by shape rather than knowledge:
|
||||
- **Randomise the correct option's position** across questions - never default
|
||||
it to first. Spread it roughly evenly over the slots.
|
||||
- **Keep all options the same depth and length.** Do not describe the correct
|
||||
one more fully than the distractors - a longer or more-detailed option is a
|
||||
dead giveaway. Trim the right answer or flesh out the wrong ones until a
|
||||
reader can't tell them apart by size.
|
||||
The tool caps a call at 4 questions, so ask in batches of 4 - but run them as
|
||||
one continuous flow: fire the next batch immediately after the previous
|
||||
returns, with no narration ("Round 2 of 3") and no grading between batches.
|
||||
The author always has an "Other" free-text escape, which is fine.
|
||||
- **`--free-text`:** present all questions in one numbered list, then say
|
||||
"Answer in one reply; number your answers. I won't grade until you're done."
|
||||
Wait for the author's answers.
|
||||
- Do not proceed to grading until every answer is in.
|
||||
|
||||
### 5. Grade
|
||||
For each question, in order:
|
||||
- Verdict: **Correct** / **Partial** / **Incorrect**.
|
||||
- The model answer in one or two sentences, citing the real `path:line`.
|
||||
- One line on the gap when Partial/Incorrect - what they missed and where to look.
|
||||
Then a **Score** (e.g. 6/8, counting Partial as half) and a one-line summary of
|
||||
the pattern (e.g. "solid on intent, shaky on the error paths").
|
||||
|
||||
### 6. Readiness verdict
|
||||
End with a clear call:
|
||||
- **Ready for review** - understanding is sound; note anything to mention to
|
||||
reviewers proactively.
|
||||
- **Study first** - list the specific files/concepts to re-read before requesting
|
||||
review, each as a clickable `path:line`. Be concrete: "re-read the null handling
|
||||
in X before you send this out."
|
||||
Keep it honest - if they'd get grilled in review on something, say so now.
|
||||
|
||||
### 7. If `--save`
|
||||
Write `pr-quiz/<branch>-scorecard.md`: the questions, their answers, your grades
|
||||
and model answers, the score, and the verdict. Don't commit it unless asked.
|
||||
|
||||
## Principles
|
||||
- **The author is the examinee, not the collaborator.** During the quiz you withhold
|
||||
answers; you're measuring them, not helping them pass.
|
||||
- **A failed quiz is a successful outcome** - it caught a gap before a human's time
|
||||
was spent. Frame it that way, not as a scolding.
|
||||
- **True to the code.** Every question, answer, and grade traces to a line you read.
|
||||
- **Terse and direct** in chat - the questions and the verdict, minimal preamble.
|
||||
@@ -9,6 +9,7 @@ ci: &ci
|
||||
build: &build
|
||||
- *ci
|
||||
- build.gradle
|
||||
- gradle/spotless.gradle
|
||||
- app/(common|core|proprietary|saas)/build.gradle
|
||||
- Taskfile.yml
|
||||
- .taskfiles/backend.yml
|
||||
@@ -83,6 +84,7 @@ frontend: &frontend
|
||||
- .taskfiles/frontend.yml
|
||||
- .taskfiles/e2e.yml
|
||||
- .github/workflows/frontend-validation.yml
|
||||
- .github/workflows/frontend-a11y.yml
|
||||
- .github/workflows/e2e-stubbed.yml
|
||||
- .github/workflows/e2e-live.yml
|
||||
|
||||
@@ -155,4 +157,5 @@ proprietary: &proprietary
|
||||
- configs/settings.yml.template
|
||||
- build.gradle
|
||||
- app/proprietary/build.gradle
|
||||
- gradle/spotless.gradle
|
||||
- .github/workflows/build-enterprise.yml
|
||||
|
||||
@@ -163,7 +163,7 @@ labels:
|
||||
- '.github/workflows/scorecards.yml'
|
||||
- 'exampleYmlFiles/test_cicd.yml'
|
||||
|
||||
- label: 'Github'
|
||||
- label: 'GitHub'
|
||||
files:
|
||||
- '.github/.*'
|
||||
|
||||
|
||||
+11
-7
@@ -5,6 +5,7 @@
|
||||
# the GitHub Action https://github.com/marketplace/actions/github-labeler.
|
||||
- name: "Licenses"
|
||||
color: "EDEDED"
|
||||
description: "Issues or pull requests related to licenses"
|
||||
from_name: "licenses"
|
||||
- name: "Back End"
|
||||
color: "20CE6C"
|
||||
@@ -146,21 +147,21 @@
|
||||
description: "Changes that do not affect the meaning of the code (formatting, etc.)"
|
||||
- name: "admin"
|
||||
color: "195055"
|
||||
- name: "codex"
|
||||
color: "ededed"
|
||||
description: null
|
||||
- name: "Github"
|
||||
- name: "GitHub"
|
||||
color: "0052CC"
|
||||
- name: "github_actions"
|
||||
color: "000000"
|
||||
description: "Pull requests that update GitHub Actions code"
|
||||
description: "Issues or pull requests related to GitHub configuration and integrations"
|
||||
from_name: "Github"
|
||||
- name: "needs-changes"
|
||||
color: "A65A86"
|
||||
description: "Pull requests that require changes before they can be merged"
|
||||
- name: "on-hold"
|
||||
color: "2526F9"
|
||||
- name: "python"
|
||||
color: "2b67c6"
|
||||
description: "Pull requests that update Python code"
|
||||
- name: "engine"
|
||||
color: "2b67c6"
|
||||
description: "Issues or pull requests related to the engine"
|
||||
- name: "size:L"
|
||||
color: "eb9500"
|
||||
description: "This PR changes 100-499 lines ignoring generated files."
|
||||
@@ -201,3 +202,6 @@
|
||||
- name: "license-review-required"
|
||||
color: "EDEDED"
|
||||
description: "This PR requires a license review"
|
||||
- name: "has conflicts"
|
||||
color: "D93F0B"
|
||||
description: "Pull request has merge conflicts with the base branch"
|
||||
|
||||
@@ -1,101 +1,116 @@
|
||||
#
|
||||
# This file is autogenerated by pip-compile with Python 3.12
|
||||
# This file is autogenerated by pip-compile with Python 3.13
|
||||
# by the following command:
|
||||
#
|
||||
# pip-compile --allow-unsafe --generate-hashes --output-file='.github\scripts\requirements_dev.txt' --strip-extras '.github\scripts\requirements_dev.in'
|
||||
#
|
||||
# WARNING: pip install will require the following package to be hashed.
|
||||
# Consider using a hashable URL like https://github.com/jazzband/pip-tools/archive/SOMECOMMIT.zip
|
||||
# CVE-2025-6176 mitigation: pin brotli to a specific commit
|
||||
brotli @ git+https://github.com/google/brotli.git@028fb5a23661f123017c060daa546b55cf4bde29
|
||||
# via
|
||||
# -r .github/scripts/requirements_dev.in
|
||||
# fonttools
|
||||
cffi==2.0.0 \
|
||||
--hash=sha256:00bdf7acc5f795150faa6957054fbbca2439db2f775ce831222b66f192f03beb \
|
||||
--hash=sha256:07b271772c100085dd28b74fa0cd81c8fb1a3ba18b21e03d7c27f3436a10606b \
|
||||
--hash=sha256:087067fa8953339c723661eda6b54bc98c5625757ea62e95eb4898ad5e776e9f \
|
||||
--hash=sha256:0a1527a803f0a659de1af2e1fd700213caba79377e27e4693648c2923da066f9 \
|
||||
--hash=sha256:0cf2d91ecc3fcc0625c2c530fe004f82c110405f101548512cce44322fa8ac44 \
|
||||
--hash=sha256:0f6084a0ea23d05d20c3edcda20c3d006f9b6f3fefeac38f59262e10cef47ee2 \
|
||||
--hash=sha256:12873ca6cb9b0f0d3a0da705d6086fe911591737a59f28b7936bdfed27c0d47c \
|
||||
--hash=sha256:19f705ada2530c1167abacb171925dd886168931e0a7b78f5bffcae5c6b5be75 \
|
||||
--hash=sha256:1cd13c99ce269b3ed80b417dcd591415d3372bcac067009b6e0f59c7d4015e65 \
|
||||
--hash=sha256:1e3a615586f05fc4065a8b22b8152f0c1b00cdbc60596d187c2a74f9e3036e4e \
|
||||
--hash=sha256:1f72fb8906754ac8a2cc3f9f5aaa298070652a0ffae577e0ea9bd480dc3c931a \
|
||||
--hash=sha256:1fc9ea04857caf665289b7a75923f2c6ed559b8298a1b8c49e59f7dd95c8481e \
|
||||
--hash=sha256:203a48d1fb583fc7d78a4c6655692963b860a417c0528492a6bc21f1aaefab25 \
|
||||
--hash=sha256:2081580ebb843f759b9f617314a24ed5738c51d2aee65d31e02f6f7a2b97707a \
|
||||
--hash=sha256:21d1152871b019407d8ac3985f6775c079416c282e431a4da6afe7aefd2bccbe \
|
||||
--hash=sha256:24b6f81f1983e6df8db3adc38562c83f7d4a0c36162885ec7f7b77c7dcbec97b \
|
||||
--hash=sha256:256f80b80ca3853f90c21b23ee78cd008713787b1b1e93eae9f3d6a7134abd91 \
|
||||
--hash=sha256:28a3a209b96630bca57cce802da70c266eb08c6e97e5afd61a75611ee6c64592 \
|
||||
--hash=sha256:2c8f814d84194c9ea681642fd164267891702542f028a15fc97d4674b6206187 \
|
||||
--hash=sha256:2de9a304e27f7596cd03d16f1b7c72219bd944e99cc52b84d0145aefb07cbd3c \
|
||||
--hash=sha256:38100abb9d1b1435bc4cc340bb4489635dc2f0da7456590877030c9b3d40b0c1 \
|
||||
--hash=sha256:3925dd22fa2b7699ed2617149842d2e6adde22b262fcbfada50e3d195e4b3a94 \
|
||||
--hash=sha256:3e17ed538242334bf70832644a32a7aae3d83b57567f9fd60a26257e992b79ba \
|
||||
--hash=sha256:3e837e369566884707ddaf85fc1744b47575005c0a229de3327f8f9a20f4efeb \
|
||||
--hash=sha256:3f4d46d8b35698056ec29bca21546e1551a205058ae1a181d871e278b0b28165 \
|
||||
--hash=sha256:44d1b5909021139fe36001ae048dbdde8214afa20200eda0f64c068cac5d5529 \
|
||||
--hash=sha256:45d5e886156860dc35862657e1494b9bae8dfa63bf56796f2fb56e1679fc0bca \
|
||||
--hash=sha256:4647afc2f90d1ddd33441e5b0e85b16b12ddec4fca55f0d9671fef036ecca27c \
|
||||
--hash=sha256:4671d9dd5ec934cb9a73e7ee9676f9362aba54f7f34910956b84d727b0d73fb6 \
|
||||
--hash=sha256:53f77cbe57044e88bbd5ed26ac1d0514d2acf0591dd6bb02a3ae37f76811b80c \
|
||||
--hash=sha256:5eda85d6d1879e692d546a078b44251cdd08dd1cfb98dfb77b670c97cee49ea0 \
|
||||
--hash=sha256:5fed36fccc0612a53f1d4d9a816b50a36702c28a2aa880cb8a122b3466638743 \
|
||||
--hash=sha256:61d028e90346df14fedc3d1e5441df818d095f3b87d286825dfcbd6459b7ef63 \
|
||||
--hash=sha256:66f011380d0e49ed280c789fbd08ff0d40968ee7b665575489afa95c98196ab5 \
|
||||
--hash=sha256:6824f87845e3396029f3820c206e459ccc91760e8fa24422f8b0c3d1731cbec5 \
|
||||
--hash=sha256:6c6c373cfc5c83a975506110d17457138c8c63016b563cc9ed6e056a82f13ce4 \
|
||||
--hash=sha256:6d02d6655b0e54f54c4ef0b94eb6be0607b70853c45ce98bd278dc7de718be5d \
|
||||
--hash=sha256:6d50360be4546678fc1b79ffe7a66265e28667840010348dd69a314145807a1b \
|
||||
--hash=sha256:730cacb21e1bdff3ce90babf007d0a0917cc3e6492f336c2f0134101e0944f93 \
|
||||
--hash=sha256:737fe7d37e1a1bffe70bd5754ea763a62a066dc5913ca57e957824b72a85e205 \
|
||||
--hash=sha256:74a03b9698e198d47562765773b4a8309919089150a0bb17d829ad7b44b60d27 \
|
||||
--hash=sha256:7553fb2090d71822f02c629afe6042c299edf91ba1bf94951165613553984512 \
|
||||
--hash=sha256:7a66c7204d8869299919db4d5069a82f1561581af12b11b3c9f48c584eb8743d \
|
||||
--hash=sha256:7cc09976e8b56f8cebd752f7113ad07752461f48a58cbba644139015ac24954c \
|
||||
--hash=sha256:81afed14892743bbe14dacb9e36d9e0e504cd204e0b165062c488942b9718037 \
|
||||
--hash=sha256:8941aaadaf67246224cee8c3803777eed332a19d909b47e29c9842ef1e79ac26 \
|
||||
--hash=sha256:89472c9762729b5ae1ad974b777416bfda4ac5642423fa93bd57a09204712322 \
|
||||
--hash=sha256:8ea985900c5c95ce9db1745f7933eeef5d314f0565b27625d9a10ec9881e1bfb \
|
||||
--hash=sha256:8eca2a813c1cb7ad4fb74d368c2ffbbb4789d377ee5bb8df98373c2cc0dee76c \
|
||||
--hash=sha256:92b68146a71df78564e4ef48af17551a5ddd142e5190cdf2c5624d0c3ff5b2e8 \
|
||||
--hash=sha256:9332088d75dc3241c702d852d4671613136d90fa6881da7d770a483fd05248b4 \
|
||||
--hash=sha256:94698a9c5f91f9d138526b48fe26a199609544591f859c870d477351dc7b2414 \
|
||||
--hash=sha256:9a67fc9e8eb39039280526379fb3a70023d77caec1852002b4da7e8b270c4dd9 \
|
||||
--hash=sha256:9de40a7b0323d889cf8d23d1ef214f565ab154443c42737dfe52ff82cf857664 \
|
||||
--hash=sha256:a05d0c237b3349096d3981b727493e22147f934b20f6f125a3eba8f994bec4a9 \
|
||||
--hash=sha256:afb8db5439b81cf9c9d0c80404b60c3cc9c3add93e114dcae767f1477cb53775 \
|
||||
--hash=sha256:b18a3ed7d5b3bd8d9ef7a8cb226502c6bf8308df1525e1cc676c3680e7176739 \
|
||||
--hash=sha256:b1e74d11748e7e98e2f426ab176d4ed720a64412b6a15054378afdb71e0f37dc \
|
||||
--hash=sha256:b21e08af67b8a103c71a250401c78d5e0893beff75e28c53c98f4de42f774062 \
|
||||
--hash=sha256:b4c854ef3adc177950a8dfc81a86f5115d2abd545751a304c5bcf2c2c7283cfe \
|
||||
--hash=sha256:b882b3df248017dba09d6b16defe9b5c407fe32fc7c65a9c69798e6175601be9 \
|
||||
--hash=sha256:baf5215e0ab74c16e2dd324e8ec067ef59e41125d3eade2b863d294fd5035c92 \
|
||||
--hash=sha256:c649e3a33450ec82378822b3dad03cc228b8f5963c0c12fc3b1e0ab940f768a5 \
|
||||
--hash=sha256:c654de545946e0db659b3400168c9ad31b5d29593291482c43e3564effbcee13 \
|
||||
--hash=sha256:c6638687455baf640e37344fe26d37c404db8b80d037c3d29f58fe8d1c3b194d \
|
||||
--hash=sha256:c8d3b5532fc71b7a77c09192b4a5a200ea992702734a2e9279a37f2478236f26 \
|
||||
--hash=sha256:cb527a79772e5ef98fb1d700678fe031e353e765d1ca2d409c92263c6d43e09f \
|
||||
--hash=sha256:cf364028c016c03078a23b503f02058f1814320a56ad535686f90565636a9495 \
|
||||
--hash=sha256:d48a880098c96020b02d5a1f7d9251308510ce8858940e6fa99ece33f610838b \
|
||||
--hash=sha256:d68b6cef7827e8641e8ef16f4494edda8b36104d79773a334beaa1e3521430f6 \
|
||||
--hash=sha256:d9b29c1f0ae438d5ee9acb31cadee00a58c46cc9c0b2f9038c6b0b3470877a8c \
|
||||
--hash=sha256:d9b97165e8aed9272a6bb17c01e3cc5871a594a446ebedc996e2397a1c1ea8ef \
|
||||
--hash=sha256:da68248800ad6320861f129cd9c1bf96ca849a2771a59e0344e88681905916f5 \
|
||||
--hash=sha256:da902562c3e9c550df360bfa53c035b2f241fed6d9aef119048073680ace4a18 \
|
||||
--hash=sha256:dbd5c7a25a7cb98f5ca55d258b103a2054f859a46ae11aaf23134f9cc0d356ad \
|
||||
--hash=sha256:dd4f05f54a52fb558f1ba9f528228066954fee3ebe629fc1660d874d040ae5a3 \
|
||||
--hash=sha256:de8dad4425a6ca6e4e5e297b27b5c824ecc7581910bf9aee86cb6835e6812aa7 \
|
||||
--hash=sha256:e11e82b744887154b182fd3e7e8512418446501191994dbf9c9fc1f32cc8efd5 \
|
||||
--hash=sha256:e6e73b9e02893c764e7e8d5bb5ce277f1a009cd5243f8228f75f842bf937c534 \
|
||||
--hash=sha256:f73b96c41e3b2adedc34a7356e64c8eb96e03a3782b535e043a986276ce12a49 \
|
||||
--hash=sha256:f93fd8e5c8c0a4aa1f424d6173f14a892044054871c771f8566e4008eaa359d2 \
|
||||
--hash=sha256:fc33c5141b55ed366cfaad382df24fe7dcbc686de5be719b207bb248e3053dc5 \
|
||||
--hash=sha256:fc7de24befaeae77ba923797c7c87834c73648a05a4bde34b3b7e5588973a453 \
|
||||
--hash=sha256:fe562eb1a64e67dd297ccc4f5addea2501664954f2692b69a76449ec7913ecbf
|
||||
cffi==2.1.0 \
|
||||
--hash=sha256:02cb7ff33ded4f1532476731f89ede53e2e488a8e6205515a82144246ffa7dcc \
|
||||
--hash=sha256:03e9810d18c646077e501f661b682fbf5dee4676048527ca3cffe66faa9960dd \
|
||||
--hash=sha256:0520e1f4c35f44e209cbbb421b67eec42e6a157f59444dfb6058874ff3610e5d \
|
||||
--hash=sha256:0582a58f3051372229ca8e7f5f589f9e5632678208d8636fea3676711fdf7fe5 \
|
||||
--hash=sha256:0611e7ebf90573a535ebdc33ae9da222d037853983e13359f580fab781ca017f \
|
||||
--hash=sha256:0a42c688d19fca6e095a53c6a6e2295a5b050a8b289f109adab02a9e61a25de6 \
|
||||
--hash=sha256:0a96b74cda968eebbad56d973efe5098974f0a9fb323865bf99ea1fd24e3e64c \
|
||||
--hash=sha256:10537b1df4967ca26d21e5072d7d54188354483b91dc75058968d3f0cf13fbda \
|
||||
--hash=sha256:11b3fb55f4f8ad92274ed26705f65d8f91457de71f5380061eb6d125a768fecd \
|
||||
--hash=sha256:15faec4adfff450819f3aee0e2e02c812de6edb88203aa58807955db2003472a \
|
||||
--hash=sha256:164bff1657b2a74f0b6d54e11c9b375bc97b931f2ca9c43fcf875838da1570dd \
|
||||
--hash=sha256:1854b724d00f6654c742097d5387569021be12d3a0f770eae1df8f8acfcc6acd \
|
||||
--hash=sha256:19c54ac121cad98450b4896fa9a43ee0180d57bc4bc911a33db6cab1efab6cd3 \
|
||||
--hash=sha256:1b96bfe2c4bd825681b7d311ad6d9b7280a091f43e8f63da5729638083cd3bfb \
|
||||
--hash=sha256:1e9f50d192a3e525b15a75ab5114e442d83d657b7ec29182a991bc9a88fd3a66 \
|
||||
--hash=sha256:1ff3456eab0d889592d1936d6125bbfbc7ae4d3354a700f8bd80450a66445d4d \
|
||||
--hash=sha256:2282cd5e38aa8accd03e99d1256af8411c84cdbee6a89d841b563fdbd1f3e50f \
|
||||
--hash=sha256:276f20fffd7b396e12516ba8edf9509210ac248cbbc5acbc39cd512f9f59ebe6 \
|
||||
--hash=sha256:2b71d409cccee78310ab5dec549aed052aaea483346e282c7b02362596e01bb0 \
|
||||
--hash=sha256:2e9dabb9abcb7ad15938c7196ad5c1718a4e6d33cc79b4c0209bdb64c4a54a5c \
|
||||
--hash=sha256:30b65779d598c370374fefabf138d456fd6f3216bfa7bedfab1ba82025b0cd93 \
|
||||
--hash=sha256:33eb1ad83ebe8f313e0df035c406227d55a79456704a863fad9842136af5ad7d \
|
||||
--hash=sha256:35aaea0c7ee0e58a5cd8c2fd1a48fdf7ece0d2699b7ecdda08194e9ce5dd9b3d \
|
||||
--hash=sha256:3681e031db29958a7502f5c0c9d6bbc4c36cb20f7b104086fa642d1799631ff8 \
|
||||
--hash=sha256:379de10ce1ba048b1448599d1b37b24caee16309d1ac98d3982fc997f768700b \
|
||||
--hash=sha256:37f525a7e7e50c017fdebe58b787be310ad59357ae43a053943a6e1a6c526001 \
|
||||
--hash=sha256:3b926723c13eba9f81d2ef3820d63aeceec3b2d4639906047bf675cb8a7a500d \
|
||||
--hash=sha256:3d7f118b5adbfdfead90c25822690b02bc8074fba949bb7858bec4ebd55adb43 \
|
||||
--hash=sha256:46b1c8db8f6122420f32d02fffb924c2fe9bc772d228c7c711748fff56aabb2b \
|
||||
--hash=sha256:47ff3a8bfd8cb9da1af7524b965127095055654c177fcfc7578debcb015eecd0 \
|
||||
--hash=sha256:4d433a51f1870e43a13b6732f92aaf540ff77c2015097c78556f75a2d6c030e0 \
|
||||
--hash=sha256:4f26194e3d95e06501b942642855aed4f953d55e95d7d01b7c4483db3ecff458 \
|
||||
--hash=sha256:510aeeeac94811b138077451da1fb18b308a5feab47dd2b603af55804155e1c8 \
|
||||
--hash=sha256:5972433ad71a9e46516584ef60a0fda12d9dc459938d1539c3ddecf9bdc1368d \
|
||||
--hash=sha256:5ecbd0499275d57506d397eebe1981cee87b47fcd9ef5c22cab7ed7644a39a94 \
|
||||
--hash=sha256:6274dcb2d15cef48daa73ed1be5a40d501d74dccd0cd6db364776d12cb6ba022 \
|
||||
--hash=sha256:63960549e4f8dc41e31accb97b975abaecfc44c03e396c093a6436763c2ea7db \
|
||||
--hash=sha256:64c753a0f87a256020004f37a1c8c02c480e725f910f0b2a0f3f07debd1b2479 \
|
||||
--hash=sha256:6af371f3767faeffc6ac1ef57cdfd25844403e9d3f476c5537caee499de96376 \
|
||||
--hash=sha256:6ca4919c6e4f89aa99c42510b42cf54596892c00b3f9077f6bdd1505e24b9c8d \
|
||||
--hash=sha256:6d194185eabd279f1c05ebe3504265ddfc5ad2b58d0714f7db9f01da592e9eb6 \
|
||||
--hash=sha256:702c436735fbe99d59ada02a1f65cfc0d31c0ee8b7290912f8fbc5cd1e4b16c3 \
|
||||
--hash=sha256:716ff8ec22f20b4d988b12884086bcef0fc99737043e503f7a3935a6be99b1ea \
|
||||
--hash=sha256:762f99479dcb369f60ab9017ad4ab97a36a1dd7c1ee5a3b15db0f4b8659120cd \
|
||||
--hash=sha256:7762faa47e8ff7eb80bd261d9a7d8eea2d8baa69de5e95b70c1f338bbe712f02 \
|
||||
--hash=sha256:78474632761faa0fb96f30b1c928c84ebcf68713cbb80d15bab09dfe61640fde \
|
||||
--hash=sha256:799416bae98336e400981ff6e532d67d5c709cfb30afb79865a1315f94b0e224 \
|
||||
--hash=sha256:7d034dcffa09e9a46c93fa3a3be402096cb5354ac6e41ab8e5cc9cd8b642ad76 \
|
||||
--hash=sha256:7d28dff1db6764108bc30788d85d61c876beff416d9a49cb9dd7c5a9f34f5804 \
|
||||
--hash=sha256:7d3538f9c0e50670f4deb93dbb696576e60590369cae2faf7de681e597a8a1f1 \
|
||||
--hash=sha256:7d5980a3433d4b71a5e120f9dd551403d7824e31e2e67124fe2769c404c06913 \
|
||||
--hash=sha256:7ea6b3e2c4250ff1de21c630fe72d0f63eb95c2c32ffbf64a358cf4a8836d714 \
|
||||
--hash=sha256:86cf8755a791f72c85dc287128cc62d4f24d392e3f1e15837245623f4a33cccc \
|
||||
--hash=sha256:88023dfe18799507b73f1dbb0d14326a17465de1bc9c9c7655c22845e9ddc3a2 \
|
||||
--hash=sha256:89095c1968b4ba8285840e131bf2891b09ae137fe2146905acae0354fbce1b5e \
|
||||
--hash=sha256:8d35c139744adb3e727cd51b1a18324bbe44b8bd41bf8322bca4d41289f48eda \
|
||||
--hash=sha256:8e74a6135550c4748af665b1b1118b6aab33b1fc6a16f9aff630af107c3b4512 \
|
||||
--hash=sha256:8f9ec95b8a043d3dfbc74d9abc6f7baf524dd27a8dc160b0a32ff9cdab650c28 \
|
||||
--hash=sha256:90bec57cf82089383bd06a605b3eb8daebf7e5a668520beaf6e327a83a947699 \
|
||||
--hash=sha256:95f2954c2c9473d892eca6e0409f3568b37ab62a8eedb122461f73cc273476e3 \
|
||||
--hash=sha256:961be50688f7fba2fa65f63712d3b9b341a22311f5253460ce933f52f0de1c8c \
|
||||
--hash=sha256:98fff996e983a36d3aa2eca83af40c5821202e7e6f32d13ae94e3d2286f10cfe \
|
||||
--hash=sha256:9b8f0f26ca4e7513c534d351eca551947d053fac438f2a04ac96d882909b0d3a \
|
||||
--hash=sha256:9d72af0cf10a76a600a9690078fe31c63b9588c8e86bf9fd353f713c84b5db0f \
|
||||
--hash=sha256:9d8272c0e483b024e1b9ad029821470ed8ec65631dbd90217469da0e7cd89f1c \
|
||||
--hash=sha256:a016194dbe13d14ee9556e734b772d8d67b947092b268d757fd4290e3ba2dfc2 \
|
||||
--hash=sha256:a5781494d4d400a3f47f8f1da94b324f6e6b440a53387774002890a2a2f4b50f \
|
||||
--hash=sha256:a95b05f9baf29b91171b3a8bd2020b028835243e7b0ff6bb23e2a3c228518b1b \
|
||||
--hash=sha256:aa7a1b53a2a4452ada2d1b5dade9960b2522f1e61293a811a077439e39029565 \
|
||||
--hash=sha256:ac0f1a2d0cfa7eea3f2aaf006ab6e70e8feeb16b75d65b7e5939982ca2f11056 \
|
||||
--hash=sha256:af5e2915d41fe6c961694d7bfdc8562942638200f3ce2765dfb8b745cf997629 \
|
||||
--hash=sha256:b6422532152adf4e59b110cb2808cee7a033800952f5c036b4af047ee43199e7 \
|
||||
--hash=sha256:b65f590ef2a44640f9a05dbb548a429b4ade77913ce683ac8b1480777658a6c0 \
|
||||
--hash=sha256:ba00f661f8ba35d075c937174e27c2c421cec3942fd2e0ea3e66996757c0fdd9 \
|
||||
--hash=sha256:bccbbb5ee76a61f9d99b5bf3846a51d7fca4b6a732fe46f89295610edaf41853 \
|
||||
--hash=sha256:bf01d8c84cbea96b944c73b22182e6c7c432b3475632b8111dbfdc95ddad6e13 \
|
||||
--hash=sha256:bf5c6cf48238b0eb4c086978c492ad1cbc22373fc5b2d7353b3a598ce6db887a \
|
||||
--hash=sha256:c16914df9fb7f500e440e6875fa23ff5e0b31db01fa9c06af98d59a91f0dc2e4 \
|
||||
--hash=sha256:c351efb95e832a853a29361675f33a7ce53de1a109cd73fd47af0712213aa4ce \
|
||||
--hash=sha256:c4165821e131d6d4ca444347c2b694e2311bcfa3fe5a861cc72968f28867beac \
|
||||
--hash=sha256:c5f5df567f6eb216de69be06ce55c8b714090fae02b18a3b40da8163b8c5fa9c \
|
||||
--hash=sha256:c941bb58d5a6e1c3892d86e42927ed6c180302f07e6d395d08c416e594b98b46 \
|
||||
--hash=sha256:c97f080ea627e2863524c5af3836e2270b5f5dfff1f104392b959f8df0c5d384 \
|
||||
--hash=sha256:cb96698e3c7413d906ce83f8ffd245ec1bd94707541f299d0ce4d6b0193e982b \
|
||||
--hash=sha256:cbb7640ce37159548d2147b5b8c241f962143d4c71231431820783f4dc78f210 \
|
||||
--hash=sha256:cdf2448aab5f661c9315308ec8b93f4e8a1a67a3c733f8631067a2b67d5913dc \
|
||||
--hash=sha256:d2117334c3af3bdcb9a88522b844a2bdb5efdc4f71c6c822df55486ae1c3347a \
|
||||
--hash=sha256:d53d10f7da99ae46f7373b9150393e9c5eab9b224909982b43832668de4779f5 \
|
||||
--hash=sha256:d9fafc5aa2e2a39aaf7f8cc0c1f044a9b07fca12e558dca53a3cc5c654ad67a7 \
|
||||
--hash=sha256:db3eb7d46527159a878ec3460e9d40615bc25ba337d477db681aea6e4f05c5d2 \
|
||||
--hash=sha256:dbf7c7a88e2bac086f06d14577332760bdeecc42bdec8ac4077f6260557d9326 \
|
||||
--hash=sha256:df2b82571a1b30f58a87bf4e5a9e78d2b1eff6c6ce8fd3aa3757221f93f0863f \
|
||||
--hash=sha256:df92f2aba50eb4d96718b68ef76f2e57a57b54f2fa62333496d16c6d585a85ca \
|
||||
--hash=sha256:eb4e8997a49aa2c08a3e43c9045d224448b8941d88e7ac163c7d383e560cbf98 \
|
||||
--hash=sha256:efc1cdd798b1aaf39b4610bba7aad28c9bea9b910f25c784ccf9ec1fa719d1f9 \
|
||||
--hash=sha256:f146d154428a2523f9cc7936c02353c2459b8f6cf07d3cd1ee1c0a611109c5d5 \
|
||||
--hash=sha256:f5bce581e6b8c235e566a14768a943b172ada3ed73537bb0c0be1edee312d4e7 \
|
||||
--hash=sha256:f9912624a0c0b834b7520d7769b3644453aabc0a7e1c839da7359f050750e9bc \
|
||||
--hash=sha256:fb62edb5bb52cca65fab91a63afa7561607120d26090a7e8fda6fb9f064726da \
|
||||
--hash=sha256:ff067a8d8d880e7809e4ac88eb009bb848870115317b306666502ccad30b147f
|
||||
# via weasyprint
|
||||
cfgv==3.5.0 \
|
||||
--hash=sha256:a8dc6b26ad22ff227d2634a65cb388215ce6cc96bbcc5cfde7641ae87e8dacc0 \
|
||||
@@ -105,67 +120,67 @@ cssselect2==0.9.0 \
|
||||
--hash=sha256:6a99e5f91f9a016a304dd929b0966ca464bcfda15177b6fb4a118fc0fb5d9563 \
|
||||
--hash=sha256:759aa22c216326356f65e62e791d66160a0f9c91d1424e8d8adc5e74dddfc6fb
|
||||
# via weasyprint
|
||||
distlib==0.4.0 \
|
||||
--hash=sha256:9659f7d87e46584a30b5780e43ac7a2143098441670ff0a49d5f9034c54a6c16 \
|
||||
--hash=sha256:feec40075be03a04501a973d81f633735b4b69f98b05450592310c0f401a4e0d
|
||||
distlib==0.4.3 \
|
||||
--hash=sha256:4b0ce306c966eb73bc3a7b6abad017c556dadd92c44701562cd528ac7fde4d5b \
|
||||
--hash=sha256:f152097224a0ae24be5a0f6bae1b9359af82133bce63f98a95f86cae1aede9ed
|
||||
# via virtualenv
|
||||
filelock==3.29.0 \
|
||||
--hash=sha256:69974355e960702e789734cb4871f884ea6fe50bd8404051a3530bc07809cf90 \
|
||||
--hash=sha256:96f5f6344709aa1572bbf631c640e4ebeeb519e08da902c39a001882f30ac258
|
||||
filelock==3.30.0 \
|
||||
--hash=sha256:1774e682dbe443bd60f9609162fc596e2c80dc84ffc2957068953406d0520090 \
|
||||
--hash=sha256:40632998f0772e64183bb819f086a1b9def6be1090cf1dcb9d45f46806ef279b
|
||||
# via
|
||||
# python-discovery
|
||||
# virtualenv
|
||||
fonttools==4.62.1 \
|
||||
--hash=sha256:0aa72c43a601cfa9273bb1ae0518f1acadc01ee181a6fc60cd758d7fdadffc04 \
|
||||
--hash=sha256:0b3ae47e8636156a9accff64c02c0924cbebad62854c4a6dbdc110cd5b4b341a \
|
||||
--hash=sha256:12859ff0b47dd20f110804c3e0d0970f7b832f561630cd879969011541a464a9 \
|
||||
--hash=sha256:149f7d84afca659d1a97e39a4778794a2f83bf344c5ee5134e09995086cc2392 \
|
||||
--hash=sha256:1596aeaddf7f78e21e68293c011316a25267b3effdaccaf4d59bc9159d681b82 \
|
||||
--hash=sha256:19177c8d96c7c36359266e571c5173bcee9157b59cfc8cb0153c5673dc5a3a7d \
|
||||
--hash=sha256:1c5c25671ce8805e0d080e2ffdeca7f1e86778c5cbfbeae86d7f866d8830517b \
|
||||
--hash=sha256:1eecc128c86c552fb963fe846ca4e011b1be053728f798185a1687502f6d398e \
|
||||
--hash=sha256:268abb1cb221e66c014acc234e872b7870d8b5d4657a83a8f4205094c32d2416 \
|
||||
--hash=sha256:2d850f66830a27b0d498ee05adb13a3781637b1826982cd7e2b3789ef0cc71ae \
|
||||
--hash=sha256:2e7abd2b1e11736f58c1de27819e1955a53267c21732e78243fa2fa2e5c1e069 \
|
||||
--hash=sha256:403d28ce06ebfc547fbcb0cb8b7f7cc2f7a2d3e1a67ba9a34b14632df9e080f9 \
|
||||
--hash=sha256:40975849bac44fb0b9253d77420c6d8b523ac4dcdcefeff6e4d706838a5b80f7 \
|
||||
--hash=sha256:486f32c8047ccd05652aba17e4a8819a3a9d78570eb8a0e3b4503142947880ed \
|
||||
--hash=sha256:49a445d2f544ce4a69338694cad575ba97b9a75fff02720da0882d1a73f12800 \
|
||||
--hash=sha256:59b372b4f0e113d3746b88985f1c796e7bf830dd54b28374cd85c2b8acd7583e \
|
||||
--hash=sha256:5a648bde915fba9da05ae98856987ca91ba832949a9e2888b48c47ef8b96c5a9 \
|
||||
--hash=sha256:5f37df1cac61d906e7b836abe356bc2f34c99d4477467755c216b72aa3dc748b \
|
||||
--hash=sha256:6706d1cb1d5e6251a97ad3c1b9347505c5615c112e66047abbef0f8545fa30d1 \
|
||||
--hash=sha256:68959f5fc58ed4599b44aad161c2837477d7f35f5f79402d97439974faebfebe \
|
||||
--hash=sha256:6acb4109f8bee00fec985c8c7afb02299e35e9c94b57287f3ea542f28bd0b0a7 \
|
||||
--hash=sha256:7487782e2113861f4ddcc07c3436450659e3caa5e470b27dc2177cade2d8e7fd \
|
||||
--hash=sha256:7aa21ff53e28a9c2157acbc44e5b401149d3c9178107130e82d74ceb500e5056 \
|
||||
--hash=sha256:7bca7a1c1faf235ffe25d4f2e555246b4750220b38de8261d94ebc5ce8a23c23 \
|
||||
--hash=sha256:8d337fdd49a79b0d51c4da87bc38169d21c3abbf0c1aa9367eff5c6656fb6dae \
|
||||
--hash=sha256:8f8fca95d3bb3208f59626a4b0ea6e526ee51f5a8ad5d91821c165903e8d9260 \
|
||||
--hash=sha256:90365821debbd7db678809c7491ca4acd1e0779b9624cdc6ddaf1f31992bf974 \
|
||||
--hash=sha256:92bb00a947e666169c99b43753c4305fc95a890a60ef3aeb2a6963e07902cc87 \
|
||||
--hash=sha256:93c316e0f5301b2adbe6a5f658634307c096fd5aae60a5b3412e4f3e1728ab24 \
|
||||
--hash=sha256:942b03094d7edbb99bdf1ae7e9090898cad7bf9030b3d21f33d7072dbcb51a53 \
|
||||
--hash=sha256:9c125ffa00c3d9003cdaaf7f2c79e6e535628093e14b5de1dccb08859b680936 \
|
||||
--hash=sha256:9dde91633f77fa576879a0c76b1d89de373cae751a98ddf0109d54e173b40f14 \
|
||||
--hash=sha256:9e7863e10b3de72376280b515d35b14f5eeed639d1aa7824f4cf06779ec65e42 \
|
||||
--hash=sha256:a24decd24d60744ee8b4679d38e88b8303d86772053afc29b19d23bb8207803c \
|
||||
--hash=sha256:a5d8825e1140f04e6c99bb7d37a9e31c172f3bc208afbe02175339e699c710e1 \
|
||||
--hash=sha256:aa69d10ed420d8121118e628ad47d86e4caa79ba37f968597b958f6cceab7eca \
|
||||
--hash=sha256:ad5cca75776cd453b1b035b530e943334957ae152a36a88a320e779d61fc980c \
|
||||
--hash=sha256:b4e0fcf265ad26e487c56cb12a42dffe7162de708762db951e1b3f755319507d \
|
||||
--hash=sha256:b820fcb92d4655513d8402d5b219f94481c4443d825b4372c75a2072aa4b357a \
|
||||
--hash=sha256:bd13b7999d59c5eb1c2b442eb2d0c427cb517a0b7a1f5798fc5c9e003f5ff782 \
|
||||
--hash=sha256:bdfe592802ef939a0e33106ea4a318eeb17822c7ee168c290273cbd5fabd746c \
|
||||
--hash=sha256:c05557a78f8fa514da0f869556eeda40887a8abc77c76ee3f74cf241778afd5a \
|
||||
--hash=sha256:c22b1014017111c401469e3acc5433e6acf6ebcc6aa9efb538a533c800971c79 \
|
||||
--hash=sha256:c9b9e288b4da2f64fd6180644221749de651703e8d0c16bd4b719533a3a7d6e3 \
|
||||
--hash=sha256:d241cdc4a67b5431c6d7f115fdf63335222414995e3a1df1a41e1182acd4bcc7 \
|
||||
--hash=sha256:e54c75fd6041f1122476776880f7c3c3295ffa31962dc6ebe2543c00dca58b5d \
|
||||
--hash=sha256:e8514f4924375f77084e81467e63238b095abda5107620f49421c368a6017ed2 \
|
||||
--hash=sha256:ee91628c08e76f77b533d65feb3fbe6d9dad699f95be51cf0d022db94089cdc4 \
|
||||
--hash=sha256:ef46db46c9447103b8f3ff91e8ba009d5fe181b1920a83757a5762551e32bb68 \
|
||||
--hash=sha256:fa1d16210b6b10a826d71bed68dd9ec24a9e218d5a5e2797f37c573e7ec215ca
|
||||
fonttools==4.63.0 \
|
||||
--hash=sha256:032038247a96c1690f9f31e377c389383c902531b085aa4e4dabd6f57f870e69 \
|
||||
--hash=sha256:063e08bd17bd5a90127a14123de0d6a952dbc847695fd98b63c043d58057f90c \
|
||||
--hash=sha256:0c18358a155d75034911c5ee397a5b44cd19dd325dbb8b35fb60bf421d6a72ac \
|
||||
--hash=sha256:0eac00b9118c3c2f87d272e45341871c5b3066baa3c86897fa634a7c3fb59096 \
|
||||
--hash=sha256:1e874792a8212b44583ea02189d9e693906b2f78b261f372f95d6c563210ac1d \
|
||||
--hash=sha256:22135da48a348785c5e2d5d2d9d6bec5ed44adacbaeb9db12d9493bf6c6bfa68 \
|
||||
--hash=sha256:22693918177bd9ceabec4736d338045f357769416fc6b0b2508eefef75b08616 \
|
||||
--hash=sha256:27fdc65af8da6f88b9c6121c47a464cbe359fcfff7ff6fc2d37a1f395d755b78 \
|
||||
--hash=sha256:2b8ae05d9eacf6081414d759c0a352769ac28ce31280d6bb8e77b03f9e3c449f \
|
||||
--hash=sha256:2c14b4fd138c4bafcca294765c547914e1aa431ae1ca94ab99d8db08c958bd3b \
|
||||
--hash=sha256:308f957cdeaf8abe4e5f2f124902ef405448af92c90f80e302a3b771c2e6116b \
|
||||
--hash=sha256:37dd23e621e3b0aef1baa70a303b80aaf38449632cfc8fd2a55fb285bbccfc02 \
|
||||
--hash=sha256:445af2eab030a16b9171ea8bdda7ebf7d96bda2df88ee182a464252f6e05e20d \
|
||||
--hash=sha256:51394295f1a51de8b5f30bdb1e1b9a4231536c7064ef5c6e211eec19fa36036f \
|
||||
--hash=sha256:58dc6bb86a78d782f00f9190ca02c119cf5bbe2807536e361e18d42019f877d8 \
|
||||
--hash=sha256:59ac449f8cca9b4ffa08d2e7bbadad87ce710d69d1eda5c3c1ce579baa987272 \
|
||||
--hash=sha256:6b2248c5decb223562f7902ff6325077a073f608ee8e33e88ad88db734eb9f49 \
|
||||
--hash=sha256:6d4741eb179121cab9eea4cb2393d24492373a260d7945006358c08cfbf45419 \
|
||||
--hash=sha256:6db5140a60a5d731d21ec076745b40a310607731b0a565b50776393188649001 \
|
||||
--hash=sha256:6e528da43bc3791085f8cb6141b1d13e459226790240340fcbb4625649238b03 \
|
||||
--hash=sha256:796f27556dbe094c4824f75ca85267e4df776c79036c8441469a4df37038c196 \
|
||||
--hash=sha256:79cdc9f567aec74a72918fd060283911406750cbc9fd28c1316023deb6ce31a9 \
|
||||
--hash=sha256:7d76edbff9014094dbf03bd2d074709dfa6ec7aba13d838c937a2b33d2d6a86e \
|
||||
--hash=sha256:7d782fac32985914c351556f68ac0855391572bcd87de50e05970d3cd4c96fc5 \
|
||||
--hash=sha256:7dd683fef0663e9f0f45cf541d788d24caa3ec9db50796b588e1757d8b3bc007 \
|
||||
--hash=sha256:85be818f5506e8a7753153def2c9550178f0ecae6a47b5e0e8dbb23f7cc90380 \
|
||||
--hash=sha256:948428a275741f0b64b113c955425a953314f4b9ab9997f73a72c83e68e569c8 \
|
||||
--hash=sha256:9ced0bd02ac751dd6319b0da88aaef24414e3b0dbc32bb4f24944821a3741a27 \
|
||||
--hash=sha256:9e12f105d2b6342c559c298afb674006bb2893afc7102dcf8a1b55b0486b4e40 \
|
||||
--hash=sha256:a8b33a82979e0a6a34ff435cc81317be1f95ec1ebb7a3a2d1c8a6a54f02ae44e \
|
||||
--hash=sha256:a9faff9e0c1f76f9fd55899d2ce785832efebab37eb8ae13995853aef178bef0 \
|
||||
--hash=sha256:af2fd1664d00a397d75f806985ddb36282091c2131a73a6485c23b4a34722263 \
|
||||
--hash=sha256:afefc1ed0a59785a7fb06ea7e1678e849c193e1e387db783579bc7b3056fcfcb \
|
||||
--hash=sha256:b1cd75a03ad8cb5bc40c90bfde68c0c47de423aa19e5c0f362b43520645eea94 \
|
||||
--hash=sha256:ba04cb5891d4c0c21b6da95eda8d7b090021508a294fff33464fc7d241e0856b \
|
||||
--hash=sha256:bf00f21eb5fb721dbaf73d1e9da6d02a1af7768f2ebcf9798be98beab8ba90f6 \
|
||||
--hash=sha256:c0425b277a59cff3d80ca42162a8de360f318438a2ac83570842a678d826d579 \
|
||||
--hash=sha256:c1aaa4b9c75798400ac043ce04d74e7830376c85095a5a6ed7cba2f17a266bf4 \
|
||||
--hash=sha256:c2a2a42198b696a6f48fad91709afb55176e66a5e566131219dba372fb7f8c59 \
|
||||
--hash=sha256:caeb583deeb5168e694b65cda8b4ee62abedfa66cf88488734466f2366b9c4e0 \
|
||||
--hash=sha256:cb014d58140a38135f16064c74c652ed57aa0b75cbf8bb59cac821f7edb5334e \
|
||||
--hash=sha256:ccf41f2efdf56994d22d73bef4ced1052161958169428d06ba9724ea9e9a64be \
|
||||
--hash=sha256:cd7e9857e5e63738b9d9fd707bc1f59c8b09e5177726d23664db393c59bb08bd \
|
||||
--hash=sha256:d76ac49f929aecaf82d83250b8347e099d7aecba0f4726c1d9b6df3b8bb5fe18 \
|
||||
--hash=sha256:d7e5c9973aa04c95650c96e5f5ad865fbf42d62079163ecfab1e01cbc2504c22 \
|
||||
--hash=sha256:dcf076a4474fe0d7367e5bbf5b052c7284fa1feca729c04176ce513521afd8a0 \
|
||||
--hash=sha256:e3297a6a4059b4acc3a1e9a8b04741f240a80044eef08ebd32e8b5bcdddce75b \
|
||||
--hash=sha256:ee08ebfa58f6e1aeff5697ab9582105bb620008c1caafb681e4c557e7483027b \
|
||||
--hash=sha256:ef3048ef05dbb552b89817713d9cac912e00d0fde4a3105c00d29e52e10c89af \
|
||||
--hash=sha256:fd1e3094f42d806d3d7c79162fc59e5910fcbe3a7360c385b8da969bc4493745
|
||||
# via weasyprint
|
||||
identify==2.6.19 \
|
||||
--hash=sha256:20e6a87f786f768c092a721ad107fc9df0eb89347be9396cadf3f4abbd1fb78a \
|
||||
@@ -175,193 +190,190 @@ nodeenv==1.10.0 \
|
||||
--hash=sha256:5bb13e3eed2923615535339b3c620e76779af4cb4c6a90deccc9e36b274d3827 \
|
||||
--hash=sha256:996c191ad80897d076bdfba80a41994c2b47c68e224c542b48feba42ba00f8bb
|
||||
# via pre-commit
|
||||
numpy==2.4.4 \
|
||||
--hash=sha256:07077278157d02f65c43b1b26a3886bce886f95d20aabd11f87932750dfb14ed \
|
||||
--hash=sha256:08f2e31ed5e6f04b118e49821397f12767934cfdd12a1ce86a058f91e004ee50 \
|
||||
--hash=sha256:0aec54fd785890ecca25a6003fd9a5aed47ad607bbac5cd64f836ad8666f4959 \
|
||||
--hash=sha256:0d35aea54ad1d420c812bfa0385c71cd7cc5bcf7c65fed95fc2cd02fe8c79827 \
|
||||
--hash=sha256:0d4e437e295f18ec29bc79daf55e8a47a9113df44d66f702f02a293d93a2d6dd \
|
||||
--hash=sha256:0dfd3f9d3adbe2920b68b5cd3d51444e13a10792ec7154cd0a2f6e74d4ab3233 \
|
||||
--hash=sha256:1378871da56ca8943c2ba674530924bb8ca40cd228358a3b5f302ad60cf875fc \
|
||||
--hash=sha256:15716cfef24d3a9762e3acdf87e27f58dc823d1348f765bbea6bef8c639bfa1b \
|
||||
--hash=sha256:19710a9ca9992d7174e9c52f643d4272dcd1558c5f7af7f6f8190f633bd651a7 \
|
||||
--hash=sha256:23cbfd4c17357c81021f21540da84ee282b9c8fba38a03b7b9d09ba6b951421e \
|
||||
--hash=sha256:2483e4584a1cb3092da4470b38866634bafb223cbcd551ee047633fd2584599a \
|
||||
--hash=sha256:27a8d92cd10f1382a67d7cf4db7ce18341b66438bdd9f691d7b0e48d104c2a9d \
|
||||
--hash=sha256:28a650663f7314afc3e6ec620f44f333c386aad9f6fc472030865dc0ebb26ee3 \
|
||||
--hash=sha256:2aa0613a5177c264ff5921051a5719d20095ea586ca88cc802c5c218d1c67d3e \
|
||||
--hash=sha256:2c194dd721e54ecad9ad387c1d35e63dce5c4450c6dc7dd5611283dda239aabb \
|
||||
--hash=sha256:2d19e6e2095506d1736b7d80595e0f252d76b89f5e715c35e06e937679ea7d7a \
|
||||
--hash=sha256:2d390634c5182175533585cc89f3608a4682ccb173cc9bb940b2881c8d6f8fa0 \
|
||||
--hash=sha256:30caa73029a225b2d40d9fae193e008e24b2026b7ee1a867b7ee8d96ca1a448e \
|
||||
--hash=sha256:42c16925aa5a02362f986765f9ebabf20de75cdefdca827d14315c568dcab113 \
|
||||
--hash=sha256:45dbed2ab436a9e826e302fcdcbe9133f9b0006e5af7168afb8963a6520da103 \
|
||||
--hash=sha256:4636de7fd195197b7535f231b5de9e4b36d2c440b6e566d2e4e4746e6af0ca93 \
|
||||
--hash=sha256:4a19d9dba1a76618dd86b164d608566f393f8ec6ac7c44f0cc879011c45e65af \
|
||||
--hash=sha256:4bbc7f303d125971f60ec0aaad5e12c62d0d2c925f0ab1273debd0e4ba37aba5 \
|
||||
--hash=sha256:4d6d57903571f86180eb98f8f0c839fa9ebbfb031356d87f1361be91e433f5b7 \
|
||||
--hash=sha256:4e874c976154687c1f71715b034739b45c7711bec81db01914770373d125e392 \
|
||||
--hash=sha256:51fc224f7ca4d92656d5a5eb315f12eb5fe2c97a66249aa7b5f562528a3be38c \
|
||||
--hash=sha256:58c8b5929fcb8287cbd6f0a3fae19c6e03a5c48402ae792962ac465224a629a4 \
|
||||
--hash=sha256:5a285b3b96f951841799528cd1f4f01cd70e7e0204b4abebac9463eecfcf2a40 \
|
||||
--hash=sha256:5c70f1cc1c4efbe316a572e2d8b9b9cc44e89b95f79ca3331553fbb63716e2bf \
|
||||
--hash=sha256:62d6b0f03b694173f9fcb1fb317f7222fd0b0b103e784c6549f5e53a27718c44 \
|
||||
--hash=sha256:6a246d5914aa1c820c9443ddcee9c02bec3e203b0c080349533fae17727dfd1b \
|
||||
--hash=sha256:6aa3236c78803afbcb255045fbef97a9e25a1f6c9888357d205ddc42f4d6eba5 \
|
||||
--hash=sha256:6bbe4eb67390b0a0265a2c25458f6b90a409d5d069f1041e6aff1e27e3d9a79e \
|
||||
--hash=sha256:715d1c092715954784bc79e1174fc2a90093dc4dc84ea15eb14dad8abdcdeb74 \
|
||||
--hash=sha256:72944b19f2324114e9dc86a159787333b77874143efcf89a5167ef83cfee8af0 \
|
||||
--hash=sha256:81f4a14bee47aec54f883e0cad2d73986640c1590eb9bfaaba7ad17394481e6e \
|
||||
--hash=sha256:846300f379b5b12cc769334464656bc882e0735d27d9726568bc932fdc49d5ec \
|
||||
--hash=sha256:86b6f55f5a352b48d7fbfd2dbc3d5b780b2d79f4d3c121f33eb6efb22e9a2015 \
|
||||
--hash=sha256:874f200b2a981c647340f841730fc3a2b54c9d940566a3c4149099591e2c4c3d \
|
||||
--hash=sha256:8a87ec22c87be071b6bdbd27920b129b94f2fc964358ce38f3822635a3e2e03d \
|
||||
--hash=sha256:8b3b60bb7cba2c8c81837661c488637eee696f59a877788a396d33150c35d842 \
|
||||
--hash=sha256:8e3ed142f2728df44263aaf5fb1f5b0b99f4070c553a0d7f033be65338329150 \
|
||||
--hash=sha256:93e15038125dc1e5345d9b5b68aa7f996ec33b98118d18c6ca0d0b7d6198b7e8 \
|
||||
--hash=sha256:989824e9faf85f96ec9c7761cd8d29c531ad857bfa1daa930cba85baaecf1a9a \
|
||||
--hash=sha256:99d838547ace2c4aace6c4f76e879ddfe02bb58a80c1549928477862b7a6d6ed \
|
||||
--hash=sha256:9b2aec6af35c113b05695ebb5749a787acd63cafc83086a05771d1e1cd1e555f \
|
||||
--hash=sha256:9c585a1790d5436a5374bac930dad6ed244c046ed91b2b2a3634eb2971d21008 \
|
||||
--hash=sha256:a7164afb23be6e37ad90b2f10426149fd75aee07ca55653d2aa41e66c4ef697e \
|
||||
--hash=sha256:ac6b31e35612a26483e20750126d30d0941f949426974cace8e6b5c58a3657b0 \
|
||||
--hash=sha256:ad2e2ef14e0b04e544ea2fa0a36463f847f113d314aa02e5b402fdf910ef309e \
|
||||
--hash=sha256:b268594bccac7d7cf5844c7732e3f20c50921d94e36d7ec9b79e9857694b1b2f \
|
||||
--hash=sha256:b5f0362dc928a6ecd9db58868fca5e48485205e3855957bdedea308f8672ea4a \
|
||||
--hash=sha256:ba1f4fc670ed79f876f70082eff4f9583c15fb9a4b89d6188412de4d18ae2f40 \
|
||||
--hash=sha256:ba203255017337d39f89bdd58417f03c4426f12beed0440cfd933cb15f8669c7 \
|
||||
--hash=sha256:c901b15172510173f5cb310eae652908340f8dede90fff9e3bf6c0d8dfd92f83 \
|
||||
--hash=sha256:c9b39d38a9bd2ae1becd7eac1303d031c5c110ad31f2b319c6e7d98b135c934d \
|
||||
--hash=sha256:d2a8490669bfe99a233298348acc2d824d496dee0e66e31b66a6022c2ad74a5c \
|
||||
--hash=sha256:dddbbd259598d7240b18c9d87c56a9d2fb3b02fe266f49a7c101532e78c1d871 \
|
||||
--hash=sha256:df3775294accfdd75f32c74ae39fcba920c9a378a2fc18a12b6820aa8c1fb502 \
|
||||
--hash=sha256:e44319a2953c738205bf3354537979eaa3998ed673395b964c1176083dd46252 \
|
||||
--hash=sha256:e4a010c27ff6f210ff4c6ef34394cd61470d01014439b192ec22552ee867f2a8 \
|
||||
--hash=sha256:e823b8b6edc81e747526f70f71a9c0a07ac4e7ad13020aa736bb7c9d67196115 \
|
||||
--hash=sha256:e892aff75639bbef0d2a2cfd55535510df26ff92f63c92cd84ef8d4ba5a5557f \
|
||||
--hash=sha256:eea7ac5d2dce4189771cedb559c738a71512768210dc4e4753b107a2048b3d0e \
|
||||
--hash=sha256:ef4059d6e5152fa1a39f888e344c73fdc926e1b2dd58c771d67b0acfbf2aa67d \
|
||||
--hash=sha256:f169b9a863d34f5d11b8698ead99febeaa17a13ca044961aa8e2662a6c7766a0 \
|
||||
--hash=sha256:f2cf083b324a467e1ab358c105f6cad5ea950f50524668a80c486ff1db24e119 \
|
||||
--hash=sha256:f8474c4241bc18b750be2abea9d7a9ec84f46ef861dbacf86a4f6e043401f79e \
|
||||
--hash=sha256:f983334aea213c99992053ede6168500e5f086ce74fbc4acc3f2b00f5762e9db \
|
||||
--hash=sha256:f9e75681b59ddaa5e659898085ae0eaea229d054f2ac0c7e563a62205a700121 \
|
||||
--hash=sha256:fbc356aae7adf9e6336d336b9c8111d390a05df88f1805573ebb0807bd06fd1d \
|
||||
--hash=sha256:fcfe2045fd2e8f3cb0ce9d4ba6dba6333b8fa05bb8a4939c908cd43322d14c7e
|
||||
numpy==2.4.6 \
|
||||
--hash=sha256:001fbb8e08d942dd57599e781f2472269ee7f2755fae407b4f67b2f0b17da3f1 \
|
||||
--hash=sha256:0280e0356c0829a18d9de1cb7eee50ec22ca639878d7240307ca0943d73cd2c4 \
|
||||
--hash=sha256:043191bfa8eab18c776647b62723ac9dddece59743b13f49b2016094129c2b3f \
|
||||
--hash=sha256:06ca2f61ec4385a07a6977c55ba998a4466c123642b4a32694d3128fce18c079 \
|
||||
--hash=sha256:0a041d3d761dc3c35cc56ce0351506a02bcbc25f7b169f652435141a17db9096 \
|
||||
--hash=sha256:0ab0a9c4ffb1a6d95ef519fe4247dba8eb6b18ad93999f76b7f657039acabd47 \
|
||||
--hash=sha256:0c9136e14ed34a9e343a31c533d78a9813a69a3148332bce5e9821cb2f996e66 \
|
||||
--hash=sha256:110f8b71aacb688ec69062bb7f6938a0f8acb01b7c1c4beb453c65b6d234584d \
|
||||
--hash=sha256:112b06a867b235ef466ed3508ddf0238050df9c727cafb5301ac385b899189a1 \
|
||||
--hash=sha256:17f9ade344e7d9b464a084d69bcf18fc691cb1db67c62ed80820bf4926d78f0e \
|
||||
--hash=sha256:1e254a00cdf42b1e4d5b3d68d33af63268d41340d8885df2ab6470f2e1500147 \
|
||||
--hash=sha256:1e978ec1e8bd0e0e4de6bb75de9d30cbb74db6b6a2bb727618613703ca0167dd \
|
||||
--hash=sha256:25c692919ac5a01f170a3bfcd62d745b24fd095c353d50812637d6fcab442e75 \
|
||||
--hash=sha256:260a5d70215b61ab4fadf5c7baacd64821842975eea312125ed3c39a6391b063 \
|
||||
--hash=sha256:2803abfebfc990042cd494d8ce2d5f82e9d847af6d35ec486923aa19dbad5e73 \
|
||||
--hash=sha256:29a287e0cf63ff528da061de6b9f64a4618da591ca1046aafc54062e40ca7eab \
|
||||
--hash=sha256:29cb7f67d10b479ff07c17d33e39f78c07f71c40ef30d63c153d340e96cd3fb4 \
|
||||
--hash=sha256:3213d622a0283a39a93d188f3cf72b26862df52fbb4ca3697f51705016523d41 \
|
||||
--hash=sha256:33111801a01c12a8a1e3721f0a9232f8cfc8ae2c6b7098167e6f623c6073f402 \
|
||||
--hash=sha256:357cc07a6d7b0b182ff02249616a03742827ebb1277546b5c7cd7f7620a45698 \
|
||||
--hash=sha256:38efbc8de75c7a0fc1ac190162d892787f3f47b57cc291231aafee36b80982b7 \
|
||||
--hash=sha256:4081eb135ac24158bd51cdfbef16f1c64df7063b1143f24731387137c092bec8 \
|
||||
--hash=sha256:40fdc1ae7125e518ea98e53e69a4ebc27e1fd50510c47b7ea130cf21e5e1d42b \
|
||||
--hash=sha256:4cfe66903cc32a9921a6733d96b19bb6abf310397581bbad89c228f5abaf0ee8 \
|
||||
--hash=sha256:511dbaf848decaaaf4b4ca48032619fb3138710c4bf7da7617765edad1ef96b0 \
|
||||
--hash=sha256:55cced7c52e981362f708ad635198e97a752dfba412cc03c23bbf3bd8d5cd662 \
|
||||
--hash=sha256:56b39e5e0622a09a25bf5baf62f4bcf0cb8a41ae6e2819cf49bbc5a74c083f91 \
|
||||
--hash=sha256:5dbbdb29840ca3d91ee0fece42fc29278886d908280bfec0a5846c6f901a3eb0 \
|
||||
--hash=sha256:5f9fb9157b4ce2971008323afe46053787b526ef624fea915b261468a8421a0f \
|
||||
--hash=sha256:6180d8b35af935aed8ece3a85e0a43f87393ae0ac87c8d2c8bd2c993f7270ef3 \
|
||||
--hash=sha256:68a5124b13fa6cc2086764a20005d30bc0548146f7f5322f02fce212ca14317f \
|
||||
--hash=sha256:68bb27509ac1b9a3443094260f6326150663b06abe40b73a2f81160623da5b67 \
|
||||
--hash=sha256:6f41ae150c4e32db4f3310cdaf64b1593a03dbabe29eec77fc9b50fe64061df6 \
|
||||
--hash=sha256:7265a2f3d436e54ef9f2b52b5c937e6be778781bd97a590319d7348f1c1ca997 \
|
||||
--hash=sha256:72fbe16c6fac95aedf5937fa873445cec2110be35d8a4e9433d7501fd98dae6b \
|
||||
--hash=sha256:7d92c3819208a60205a12a245c91ad70cb0a85336659b19b834205573ac8456e \
|
||||
--hash=sha256:8155154c7c691289fe18f510b5d4657c68c67989f293f0535a91360392ff6538 \
|
||||
--hash=sha256:81a1cca95ed5bb92aa8b10dd2cdc9a0d3853a50fad926c28b5d7e8ea54389627 \
|
||||
--hash=sha256:89cd468399cfd2504718f0ba50e410dca55a170b61a02ad92bb18c8a65186e93 \
|
||||
--hash=sha256:8ad03c0965fb3c692200e74d458ca28c1dbb4ce96f9a479a8aa041ad5fabca02 \
|
||||
--hash=sha256:90f9849678c75fe7afa2d348ac842c168b0a4d3d61919687216dfc547976d853 \
|
||||
--hash=sha256:948424b06129ce883307e8cff868c31396d8dc7630a59c61d70d98dbe70f222c \
|
||||
--hash=sha256:9cd5ffd25db4e7ba6a375693b3fc0fc1791ec636c17db3720da19bde7180ec43 \
|
||||
--hash=sha256:a0df0043bdb289bde1f62da130d20df23d58b45429f752bc7a8fc5325a225ecd \
|
||||
--hash=sha256:a2c306dea656c12c68f51f4cea133cbe78ca7435eb28c735eac1d3ebe73be6e8 \
|
||||
--hash=sha256:a7830bab239b79cda9c08c2da014761cafb48da6150e1da17ac06283f43b6089 \
|
||||
--hash=sha256:a7c711e21628b52034bb5ab8d1bce291f752fcc5e92accc615778acee1ff4778 \
|
||||
--hash=sha256:aaf159caa35993cb1f56fb9b8e4610d35758e7ca005412eb1daa856a78c9c4b1 \
|
||||
--hash=sha256:ae506e6902902557576a26ff33eda8695e7ecb3cb36c3b573a0765dee114ebdb \
|
||||
--hash=sha256:b507f5c4c1d508876d1819b6bf9a49d365b96320b5d4993426b33a23ca4b8261 \
|
||||
--hash=sha256:bf162abab1c1a736333192707cef898e735a5ca00f38f27eeedf44b39d9e85eb \
|
||||
--hash=sha256:c1a2af6c6ef86344a6b0db6b97834208bf598db514f2b155042439b62605601a \
|
||||
--hash=sha256:c2d37ab77531417474168eb79d6d80b14f821a966818505d03013d0833edb7a8 \
|
||||
--hash=sha256:c4fc99836233ea196540b17ab0983aff60ed07941751930f5f4d05bc3b3b7359 \
|
||||
--hash=sha256:d581b735e177fdcdce6fed8e7e8880a3fb6ee4e3653a3ac6af01c6f4c03effc5 \
|
||||
--hash=sha256:d6da64deb6b8ed903e7560180a92f2d804ee1ba5eeb849ac2748b8c1aba1f6d7 \
|
||||
--hash=sha256:d8e8286dd7cea7895157318d1b91cdacac64c479f3cbc8dce548331728484751 \
|
||||
--hash=sha256:ddea102b48f9e339f3948bf22040944184627a30fdf7f858667673b9c5f033c8 \
|
||||
--hash=sha256:dfa20cc6ca228e6b155b11da03825975ce66aea520985dbbddf0f2a5a495c605 \
|
||||
--hash=sha256:e3e5193ef5a3dc73bceee50f7fdc2c90dbb76c42df8d8fae3d1067a583df579e \
|
||||
--hash=sha256:e3eeb0aabd6bd5ce64faae67e9935203a6991b4bc2a485a767fbafb2c5125f45 \
|
||||
--hash=sha256:e5805d5a22fd19c8ccff10a9561f9df94436b0545619ea579db2d3c35294bce2 \
|
||||
--hash=sha256:e85b752a1e912b70eaad4fafbd4d1238007ab221de2009b9a2f5ae7461239895 \
|
||||
--hash=sha256:eaf7fa2de5c0be8ae6ff8e9bea2ccd725e980541244521d8d4b5f3354a27babe \
|
||||
--hash=sha256:ebfb099f8dcf083deef3ac1ca4c1503f387cf76296fcb3816b66f5ecb5f54fdb \
|
||||
--hash=sha256:ece3d2cfe132e7d51f44a832b303895e6f2d499c5e74dfbdb06ee246147a304a \
|
||||
--hash=sha256:ed9749eef4cbd126da3dc1d6bcb3a57f5eb7ac6a6484146bdbf743f552dfc577 \
|
||||
--hash=sha256:ede83e07a75dd06bc501566c1eca2afc0d61677c1472ac9ad93fdee6e638a48d \
|
||||
--hash=sha256:ef4aea96ce4d3b074422cb4f2f64e216bf9e213004bb58ecfdf50ea02ea8eb9a \
|
||||
--hash=sha256:f3a3570c4a2a16746ac2c31a7c7c7b0c186b95ce902e33db6f28094ed7387dda \
|
||||
--hash=sha256:f407cb6b8e9d6d8c626bc73c945db1706035af8fd632295547bf1c9e46d092d6 \
|
||||
--hash=sha256:f74a575920ab21fe304421a3fc28793d82e299cae9eccb37084e9fc7f3617c20
|
||||
# via opencv-python-headless
|
||||
opencv-python-headless==4.13.0.92 \
|
||||
--hash=sha256:0525a3d2c0b46c611e2130b5fdebc94cf404845d8fa64d2f3a3b679572a5bd22 \
|
||||
--hash=sha256:0bd48544f77c68b2941392fcdf9bcd2b9cdf00e98cb8c29b2455d194763cf99e \
|
||||
--hash=sha256:1a7d040ac656c11b8c38677cc8cccdc149f98535089dbe5b081e80a4e5903209 \
|
||||
--hash=sha256:3e0a6f0a37994ec6ce5f59e936be21d5d6384a4556f2d2da9c2f9c5dc948394c \
|
||||
--hash=sha256:5c8cfc8e87ed452b5cecb9419473ee5560a989859fe1d10d1ce11ae87b09a2cb \
|
||||
--hash=sha256:77a82fe35ddcec0f62c15f2ba8a12ecc2ed4207c17b0902c7a3151ae29f37fb6 \
|
||||
--hash=sha256:a7cf08e5b191f4ebb530791acc0825a7986e0d0dee2a3c491184bd8599848a4b \
|
||||
--hash=sha256:eb60e36b237b1ebd40a912da5384b348df8ed534f6f644d8e0b4f103e272ba7d
|
||||
opencv-python-headless==5.0.0.93 \
|
||||
--hash=sha256:030ca5e0837a2963ab36ef896baa9767eb8d2b83353fb28af5a521e40dd8756f \
|
||||
--hash=sha256:09a872a157c1376ab922a69bbf22f9a95bcc7b658a9d8b436a60212b02b2eeb4 \
|
||||
--hash=sha256:10818d91510e05c04568ae12b5cd120779c70c01bf897b001a6221fe430df80f \
|
||||
--hash=sha256:1e55af3abfb462eeeabe5c775f12bdb36216d8a93a3583d69e6bd6e1d6ba7d00 \
|
||||
--hash=sha256:829717b6a95554f273e49e357cee3b3a2a26b6f4842fbc1bed2b45bdd8f87e0e \
|
||||
--hash=sha256:840bd717c21e5c11cadadc022a823315ea417f961213d06b4df010e019eb16f4 \
|
||||
--hash=sha256:b82f9831daab90b725c7c1ee1b36cb5732c367096ac76d119e64e14eb70d5f3c \
|
||||
--hash=sha256:c6bcd96b185975ea240d22cfdb15a1f6d080cc95264cfbe2621f21bb144d89b9 \
|
||||
--hash=sha256:ed709fdf9aa0bd1f2ed8549e71d19449b03a675bb581eb292285f6861953be37
|
||||
# via -r .github/scripts/requirements_dev.in
|
||||
pdf2image==1.17.0 \
|
||||
--hash=sha256:eaa959bc116b420dd7ec415fcae49b98100dda3dd18cd2fdfa86d09f112f6d57 \
|
||||
--hash=sha256:ecdd58d7afb810dffe21ef2b1bbc057ef434dabbac6c33778a38a3f7744a27e2
|
||||
# via -r .github/scripts/requirements_dev.in
|
||||
pillow==12.2.0 \
|
||||
--hash=sha256:00a2865911330191c0b818c59103b58a5e697cae67042366970a6b6f1b20b7f9 \
|
||||
--hash=sha256:01afa7cf67f74f09523699b4e88c73fb55c13346d212a59a2db1f86b0a63e8c5 \
|
||||
--hash=sha256:03e7e372d5240cc23e9f07deca4d775c0817bffc641b01e9c3af208dbd300987 \
|
||||
--hash=sha256:03f6fab9219220f041c74aeaa2939ff0062bd5c364ba9ce037197f4c6d498cd9 \
|
||||
--hash=sha256:042db20a421b9bafecc4b84a8b6e444686bd9d836c7fd24542db3e7df7baad9b \
|
||||
--hash=sha256:0538bd5e05efec03ae613fd89c4ce0368ecd2ba239cc25b9f9be7ed426b0af1f \
|
||||
--hash=sha256:0a34329707af4f73cf1782a36cd2289c0368880654a2c11f027bcee9052d35dd \
|
||||
--hash=sha256:0c838a5125cee37e68edec915651521191cef1e6aa336b855f495766e77a366e \
|
||||
--hash=sha256:144748b3af2d1b358d41286056d0003f47cb339b8c43a9ea42f5fea4d8c66b6e \
|
||||
--hash=sha256:1610dd6c61621ae1cf811bef44d77e149ce3f7b95afe66a4512f8c59f25d9ebe \
|
||||
--hash=sha256:1e1757442ed87f4912397c6d35a0db6a7b52592156014706f17658ff58bbf795 \
|
||||
--hash=sha256:22db17c68434de69d8ecfc2fe821569195c0c373b25cccb9cbdacf2c6e53c601 \
|
||||
--hash=sha256:25373b66e0dd5905ed63fa3cae13c82fbddf3079f2c8bf15c6fb6a35586324c1 \
|
||||
--hash=sha256:2bb4a8d594eacdfc59d9e5ad972aa8afdd48d584ffd5f13a937a664c3e7db0ed \
|
||||
--hash=sha256:2c727a6d53cb0018aadd8018c2b938376af27914a68a492f59dfcaca650d5eea \
|
||||
--hash=sha256:2d192a155bbcec180f8564f693e6fd9bccff5a7af9b32e2e4bf8c9c69dbad6b5 \
|
||||
--hash=sha256:2e589959f10d9824d39b350472b92f0ce3b443c0a3442ebf41c40cb8361c5b97 \
|
||||
--hash=sha256:2e5a76d03a6c6dcef67edabda7a52494afa4035021a79c8558e14af25313d453 \
|
||||
--hash=sha256:325ca0528c6788d2a6c3d40e3568639398137346c3d6e66bb61db96b96511c98 \
|
||||
--hash=sha256:34c0d99ecccea270c04882cb3b86e7b57296079c9a4aff88cb3b33563d95afaa \
|
||||
--hash=sha256:390ede346628ccc626e5730107cde16c42d3836b89662a115a921f28440e6a3b \
|
||||
--hash=sha256:394167b21da716608eac917c60aa9b969421b5dcbbe02ae7f013e7b85811c69d \
|
||||
--hash=sha256:3997232e10d2920a68d25191392e3a4487d8183039e1c74c2297f00ed1c50705 \
|
||||
--hash=sha256:3adc9215e8be0448ed6e814966ecf3d9952f0ea40eb14e89a102b87f450660d8 \
|
||||
--hash=sha256:3e080565d8d7c671db5802eedfb438e5565ffa40115216eabb8cd52d0ecce024 \
|
||||
--hash=sha256:4a6c9fa44005fa37a91ebfc95d081e8079757d2e904b27103f4f5fa6f0bf78c0 \
|
||||
--hash=sha256:4bfd07bc812fbd20395212969e41931001fd59eb55a60658b0e5710872e95286 \
|
||||
--hash=sha256:4e6c62e9d237e9b65fac06857d511e90d8461a32adcc1b9065ea0c0fa3a28150 \
|
||||
--hash=sha256:50d8520da2a6ce0af445fa6d648c4273c3eeefbc32d7ce049f22e8b5c3daecc2 \
|
||||
--hash=sha256:51c4167c34b0d8ba05b547a3bb23578d0ba17b80a5593f93bd8ecb123dd336a3 \
|
||||
--hash=sha256:56a3f9c60a13133a98ecff6197af34d7824de9b7b38c3654861a725c970c197b \
|
||||
--hash=sha256:56b25336f502b6ed02e889f4ece894a72612fe885889a6e8c4c80239ff6e5f5f \
|
||||
--hash=sha256:57850958fe9c751670e49b2cecf6294acc99e562531f4bd317fa5ddee2068463 \
|
||||
--hash=sha256:58f62cc0f00fd29e64b29f4fd923ffdb3859c9f9e6105bfc37ba1d08994e8940 \
|
||||
--hash=sha256:5c0a9f29ca8e79f09de89293f82fc9b0270bb4af1d58bc98f540cc4aedf03166 \
|
||||
--hash=sha256:5cdfebd752ec52bf5bb4e35d9c64b40826bc5b40a13df7c3cda20a2c03a0f5ed \
|
||||
--hash=sha256:5d04bfa02cc2d23b497d1e90a0f927070043f6cbf303e738300532379a4b4e0f \
|
||||
--hash=sha256:5d2fd0fa6b5d9d1de415060363433f28da8b1526c1c129020435e186794b3795 \
|
||||
--hash=sha256:62f5409336adb0663b7caa0da5c7d9e7bdbaae9ce761d34669420c2a801b2780 \
|
||||
--hash=sha256:632ff19b2778e43162304d50da0181ce24ac5bb8180122cbe1bf4673428328c7 \
|
||||
--hash=sha256:6562ace0d3fb5f20ed7290f1f929cae41b25ae29528f2af1722966a0a02e2aa1 \
|
||||
--hash=sha256:673aa32138f3e7531ccdbca7b3901dba9b70940a19ccecc6a37c77d5fdeb05b5 \
|
||||
--hash=sha256:6a6e67ea2e6feda684ed370f9a1c52e7a243631c025ba42149a2cc5934dec295 \
|
||||
--hash=sha256:6a9adfc6d24b10f89588096364cc726174118c62130c817c2837c60cf08a392b \
|
||||
--hash=sha256:6bb77b2dcb06b20f9f4b4a8454caa581cd4dd0643a08bacf821216a16d9c8354 \
|
||||
--hash=sha256:6e6b2a0c538fc200b38ff9eb6628228b77908c319a005815f2dde585a0664b60 \
|
||||
--hash=sha256:71cde9a1e1551df7d34a25462fc60325e8a11a82cc2e2f54578e5e9a1e153d65 \
|
||||
--hash=sha256:7371b48c4fa448d20d2714c9a1f775a81155050d383333e0a6c15b1123dda005 \
|
||||
--hash=sha256:766cef22385fa1091258ad7e6216792b156dc16d8d3fa607e7545b2b72061f1c \
|
||||
--hash=sha256:7b14cc0106cd9aecda615dd6903840a058b4700fcb817687d0ee4fc8b6e389be \
|
||||
--hash=sha256:7f84204dee22a783350679a0333981df803dac21a0190d706a50475e361c93f5 \
|
||||
--hash=sha256:8023abc91fba39036dbce14a7d6535632f99c0b857807cbbbf21ecc9f4717f06 \
|
||||
--hash=sha256:80b2da48193b2f33ed0c32c38140f9d3186583ce7d516526d462645fd98660ae \
|
||||
--hash=sha256:8297651f5b5679c19968abefd6bb84d95fe30ef712eb1b2d9b2d31ca61267f4c \
|
||||
--hash=sha256:88d387ff40b3ff7c274947ed3125dedf5262ec6919d83946753b5f3d7c67ea4c \
|
||||
--hash=sha256:88ddbc66737e277852913bd1e07c150cc7bb124539f94c4e2df5344494e0a612 \
|
||||
--hash=sha256:8bd7903a5f2a4545f6fd5935c90058b89d30045568985a71c79f5fd6edf9b91e \
|
||||
--hash=sha256:8be29e59487a79f173507c30ddf57e733a357f67881430449bb32614075a40ab \
|
||||
--hash=sha256:8c984051042858021a54926eb597d6ee3012393ce9c181814115df4c60b9a808 \
|
||||
--hash=sha256:8cbeb542b2ebc6fcdacabf8aca8c1a97c9b3ad3927d46b8723f9d4f033288a0f \
|
||||
--hash=sha256:8e9c4f5b3c546fa3458a29ab22646c1c6c787ea8f5ef51300e5a60300736905e \
|
||||
--hash=sha256:90e6f81de50ad6b534cab6e5aef77ff6e37722b2f5d908686f4a5c9eba17a909 \
|
||||
--hash=sha256:975385f4776fafde056abb318f612ef6285b10a1f12b8570f3647ad0d74b48ec \
|
||||
--hash=sha256:9a8a34cc89c67a65ea7437ce257cea81a9dad65b29805f3ecee8c8fe8ff25ffe \
|
||||
--hash=sha256:9aba9a17b623ef750a4d11b742cbafffeb48a869821252b30ee21b5e91392c50 \
|
||||
--hash=sha256:9f08483a632889536b8139663db60f6724bfcb443c96f1b18855860d7d5c0fd4 \
|
||||
--hash=sha256:a4e8f36e677d3336f35089648c8955c51c6d386a13cf6ee9c189c5f5bd713a9f \
|
||||
--hash=sha256:a52edc8bfff4429aaabdf4d9ee0daadbbf8562364f940937b941f87a4290f5ff \
|
||||
--hash=sha256:a830b1a40919539d07806aa58e1b114df53ddd43213d9c8b75847eee6c0182b5 \
|
||||
--hash=sha256:aa88ccfe4e32d362816319ed727a004423aab09c5cea43c01a4b435643fa34eb \
|
||||
--hash=sha256:af73337013e0b3b46f175e79492d96845b16126ddf79c438d7ea7ff27783a414 \
|
||||
--hash=sha256:b1c1fbd8a5a1af3412a0810d060a78b5136ec0836c8a4ef9aa11807f2a22f4e1 \
|
||||
--hash=sha256:b85f66ae9eb53e860a873b858b789217ba505e5e405a24b85c0464822fe88032 \
|
||||
--hash=sha256:b86024e52a1b269467a802258c25521e6d742349d760728092e1bc2d135b4d76 \
|
||||
--hash=sha256:bd9c0c7a0c681a347b3194c500cb1e6ca9cab053ea4d82a5cf45b6b754560136 \
|
||||
--hash=sha256:bfa9c230d2fe991bed5318a5f119bd6780cda2915cca595393649fc118ab895e \
|
||||
--hash=sha256:d362d1878f00c142b7e1a16e6e5e780f02be8195123f164edf7eddd911eefe7c \
|
||||
--hash=sha256:d5d38f1411c0ed9f97bcb49b7bd59b6b7c314e0e27420e34d99d844b9ce3b6f3 \
|
||||
--hash=sha256:dac8d77255a37e81a2efcbd1fc05f1c15ee82200e6c240d7e127e25e365c39ea \
|
||||
--hash=sha256:dd025009355c926a84a612fecf58bb315a3f6814b17ead51a8e48d3823d9087f \
|
||||
--hash=sha256:deede7c263feb25dba4e82ea23058a235dcc2fe1f6021025dc71f2b618e26104 \
|
||||
--hash=sha256:e74473c875d78b8e9d5da2a70f7099549f9eb37ded4e2f6a463e60125bccd176 \
|
||||
--hash=sha256:ee3120ae9dff32f121610bb08e4313be87e03efeadfc6c0d18f89127e24d0c24 \
|
||||
--hash=sha256:eedf4b74eda2b5a4b2b2fb4c006d6295df3bf29e459e198c90ea48e130dc75c3 \
|
||||
--hash=sha256:efd8c21c98c5cc60653bcb311bef2ce0401642b7ce9d09e03a7da87c878289d4 \
|
||||
--hash=sha256:f1c943e96e85df3d3478f7b691f229887e143f81fedab9b20205349ab04d73ed \
|
||||
--hash=sha256:f278f034eb75b4e8a13a54a876cc4a5ab39173d2cdd93a638e1b467fc545ac43 \
|
||||
--hash=sha256:f3f40b3c5a968281fd507d519e444c35f0ff171237f4fdde090dd60699458421 \
|
||||
--hash=sha256:f490f9368b6fc026f021db16d7ec2fbf7d89e2edb42e8ec09d2c60505f5729c7 \
|
||||
--hash=sha256:fb043ee2f06b41473269765c2feae53fc2e2fbf96e5e22ca94fb5ad677856f06 \
|
||||
--hash=sha256:fc3d34d4a8fbec3e88a79b92e5465e0f9b842b628675850d860b8bd300b159f5
|
||||
pillow==12.3.0 \
|
||||
--hash=sha256:00808c5e14ef63ac5161091d242999076604ff74b883423a11e5d7bbb38bf756 \
|
||||
--hash=sha256:04f01d28a6aaff387bf842a13be313df23ba0597a44f1a976c9feb3c6ff4711a \
|
||||
--hash=sha256:06ff022112bc9cbf83b60f8e028d94ad87b60621706487e65f673de61610ab59 \
|
||||
--hash=sha256:0740a512dc522224c77d9aa5a8d70d8b7d73fb91f2c21125d8d025d3b8990e45 \
|
||||
--hash=sha256:0847a763afefb695bc912d7c131e7e0632d4edc1d8698f58ddabec8e46b8b6d3 \
|
||||
--hash=sha256:0dd2064cbc55aaec028ef5fbb60fa47bb6c3e7918e07ff17935284b227a9d2df \
|
||||
--hash=sha256:0feb2e9d6ad6c9e3c06effe9d00f3f1e618a6643273576b016f591e9315a7139 \
|
||||
--hash=sha256:10e41f0fbf1eec8cfd234b8fe17a4caac7c9d0db4c204d3c173a8f9f6ef3232b \
|
||||
--hash=sha256:1182d52bc2d5e5d7d0949503aa7e36d12f42205dc287e4883f407b1988820d39 \
|
||||
--hash=sha256:164b31cd1a0490ab6efae01aa5df49da7061be0af1b30e035b6e9a1bfe34ee6e \
|
||||
--hash=sha256:1657923d2d45afb66526e5b933e5b3052e6bdea196c90d3abb2424e18c77dae8 \
|
||||
--hash=sha256:186941b6aef820ad110fb01fb06eb925374dc3a21b17e37ec9a53b250c6fe2d1 \
|
||||
--hash=sha256:1cca606cd25738df4ed873d5ad46bbdb3d83b5cbca291f6b4ff13a4df6b0bbe8 \
|
||||
--hash=sha256:21900ce7ba264168cd50defae43cd75d25c833ad4ad6e73ffc5596d12e25ac89 \
|
||||
--hash=sha256:236ff70b9312fb68943c703aa842ca6a758abfa45ac187a5e7c1452e96ef72b5 \
|
||||
--hash=sha256:23aceaa007d6172b02c277f0cd359c79492bbb14f7072b4ede9fbcaf20648130 \
|
||||
--hash=sha256:23d27a3e0307ec2244cc51e7287b919aa68d097504ebe19df4e76a98a3eea5bd \
|
||||
--hash=sha256:24870b09b224f7ae3c39ed07d10e819d06f8720bc551847b1d623832b5b0e28d \
|
||||
--hash=sha256:251bf95b67017e27b13d82f5b326234ca62d70f9cf4c2b9032de2358a3b12c7b \
|
||||
--hash=sha256:25b9b82bb22e6e2b3cd07b39c68b7b862001226cb3dff7130d1cb914121b39ed \
|
||||
--hash=sha256:28ce87c5ab450a9dd970b52e5aca5fe63ed432d18a2eaddd1979a00a1ba24ace \
|
||||
--hash=sha256:300557495eb45ebb8aec96c2da9c4be642fbf7cd937278b4013ba894ea8eb0eb \
|
||||
--hash=sha256:30f2aa603c41533cc25c05acd0da21636e84a315768feb631c937177db558931 \
|
||||
--hash=sha256:331b624368d4f1d069149002f25f44bc61c8919ce8ddb3c45bdad8f6e2d89510 \
|
||||
--hash=sha256:37d6d0a00072fd2948eb22bce7e1475f34569d90c87c59f7a2ec59541b77f7a6 \
|
||||
--hash=sha256:37dc8f7bbb66efe481bb60defacef820c950c24713fb44962ed6aa2a50966de1 \
|
||||
--hash=sha256:3b8182a766685eaa002637e28b4ec8d6b18819a0c71f579bf0dbaa5830297cce \
|
||||
--hash=sha256:3edce1d53195db527e0191f84b71d02022de0540bf43a16ed734ed7537b07385 \
|
||||
--hash=sha256:446c34dcc4324b084a53b705127dc15717b22c5e140ae0a3c38349d4efec071e \
|
||||
--hash=sha256:4998562bf62a445225f22e07c896bb04b35b1b1f2eb6d760584c9c51d7a5f78c \
|
||||
--hash=sha256:4b0a7fe987b14c31ebda6083f74f22b561fd3739bc0ac51e019622e3d72668c7 \
|
||||
--hash=sha256:4e8c2a84d977f50b9daed6eeaf3baef67d00d5d74d932288f02cb94518ee3ace \
|
||||
--hash=sha256:4f883547d4b7f0495ebe7056b0cc2aea76094e7a4abc8e933540f3271df27d9c \
|
||||
--hash=sha256:514435a37670e3e5e08f3945b68718b6ed329bb84367777e16f9f4dfe1e61a0f \
|
||||
--hash=sha256:53aa02d20d10c3d814d536aa4e5ac9b84ca0ff5a88377963b085ad6822f93e64 \
|
||||
--hash=sha256:5594fc43d548a7ed94949d139aa1341b270f1863f11cfd37f5a6c8b778a6b67f \
|
||||
--hash=sha256:571b9fcb07b97ef3a492028fb3d2dc0993ca23a06138b0315286566d29ef718a \
|
||||
--hash=sha256:57b3d78c95ba9059768b10e28b813002261d3f3dfc55cc48b0c988f625175827 \
|
||||
--hash=sha256:5afb51d599ea772b8365ae807ae557f18bccfe46ab261fd1c2a9ed700fc6eb17 \
|
||||
--hash=sha256:6b02afb9b97f65fbca5f31db6a2a3ba21aa93030225f150fa3f249717e938fb4 \
|
||||
--hash=sha256:6c0016e7b354317c4e9e525b937ac8596c38d2d232b419529b9cd7a1cd46e39a \
|
||||
--hash=sha256:71d6097b330eea8fd15097780c8e89cb1a8ce7838669f48c5bacd6f663dd4701 \
|
||||
--hash=sha256:756c768d0c9c2955feb7a56c37ea24aea2e369f8d36a88da270b6a9f19e62b5e \
|
||||
--hash=sha256:78cb2c6865a35ab8ff8b75fd122f6033b92a62c82801110e48ddd6c936a45d91 \
|
||||
--hash=sha256:7a743ff716f746fc19a9557f60dab1600d4613255f8a7aeb3cdde4db7eb15a66 \
|
||||
--hash=sha256:85f998ea1848bc6757289e739cfbdda3a04adfd58b02fc018ce54d754a5ce468 \
|
||||
--hash=sha256:8728f216dcdb6e6d555cf971cb34076139ad74b31fc2c14da4fafc741c5f6217 \
|
||||
--hash=sha256:877c3f311ff35410f690861c4409e7ccbf0cd2f878e50628a28e5a0bb689e658 \
|
||||
--hash=sha256:8cd2f7bdda092d99c9fc2fb7391354f306d01443d22785d0cbfafa2e2c8bb418 \
|
||||
--hash=sha256:8e95e1385e4998ae9694eeaa4730ba5457ff61185b3a55e2e7bea0880aef452a \
|
||||
--hash=sha256:962864dc93511324d51ddbb5b9f8731bf71675b93ca612a07441896f4688fb8c \
|
||||
--hash=sha256:9cf95fe4d0f84c82d282745d9bb08ad9f926efa00be4697e767b814ce40d4330 \
|
||||
--hash=sha256:9e881fca225083806662a5c43d627d215f258ff43c890f831966c7d7ba9c7402 \
|
||||
--hash=sha256:a2b55dd6b2a4c4b7d87ffa56bdb33fdc5fdb9a462173861a7bc097f17d91cb09 \
|
||||
--hash=sha256:a45650e8ce7fafffd731db8550230db6b0d306d181a90b67d3e6bca2f1990930 \
|
||||
--hash=sha256:a876864214e136f0eb367788dbd7df045f4806801518e2cfe9e13229cfe06d8f \
|
||||
--hash=sha256:ae26d61dfa7a47befdc7572b521024e8745f3d809bd95ca9505a7bba9ef849ec \
|
||||
--hash=sha256:af8d94b0db561cf68b88a267c5c44b49e134f525d0dc2cb7ed413a66bc23559a \
|
||||
--hash=sha256:b343699e8308bdc51978310e1c959c584e7869cc8c40780058c87da7781a1e94 \
|
||||
--hash=sha256:b3c777e849237620b022f7f297dd67705f9f5cf1685f09f02e46f93e92725468 \
|
||||
--hash=sha256:b629de27fda84b42cde7edef0d85f13b958b47f6e9bbcbba9b673c562a89bd8b \
|
||||
--hash=sha256:ba09209fbe443b4acccebe845d8a138b89a8f4fbaeedd44953490b5315d5e965 \
|
||||
--hash=sha256:ba54cfebe86920a559a7c4d6b9050791c20513650a1952ebe3368c7dc70306f8 \
|
||||
--hash=sha256:bcb46e2f9feff8d06323983bd83ed00c201fdcab3d74973e7072a889b3979fcd \
|
||||
--hash=sha256:bcc33feacfaefce60c12fd500a277533bdc02b10a19f7f6d348763d8140bbba7 \
|
||||
--hash=sha256:bf16ba1b4d0b6b7c8e534936632270cf70eb00dbe09005bc345b2677b726855c \
|
||||
--hash=sha256:cf1845d02ad822a369a49f2bb9345b1614744267682e7a03527dc3bf6eea1777 \
|
||||
--hash=sha256:d69141514cc30b774ceea5e3ed3a6635c8d8a96edf664689b890f4089111fb35 \
|
||||
--hash=sha256:d9c7f76c0673154f044e9d78c8655fb4213f6ca31a836df48b40fe5d187717b9 \
|
||||
--hash=sha256:dbce0b29841537a2fa4a214c2bbf14de3587c9680caa9b4e217568472490b28f \
|
||||
--hash=sha256:dc624f6bc473dacdf7ef7eb8678d0d08edf15cd94fad6ae5c7d6cc67a4e4902f \
|
||||
--hash=sha256:e158cb00350dc278f3b91551101aa7d12415a66ebf2c91d8d5ac14e56ddd3ad0 \
|
||||
--hash=sha256:e491916b378fba47242221bb9ead245211b70d504f495d105d17b14a24b4907c \
|
||||
--hash=sha256:e795b7eb908249c4e43c7c99fac7c2c75dab0c43566e37db472a355f63693d71 \
|
||||
--hash=sha256:e7e480451b9fa137494bccd3a7d69adbe8ac65a87d97be61e11f1b1050a5bac3 \
|
||||
--hash=sha256:e91206ee562682b51b98ef4b26a6ef48fd84e15fd4c4bc5ec768eb641d206838 \
|
||||
--hash=sha256:e9871b1ffbfa9656b60aeee92ed5136a5742696006fa322b29ea3d8da0ecc9cf \
|
||||
--hash=sha256:e9aeb04d6aef139de265b29683e119b638208f88cf73cdd1658aa07221165321 \
|
||||
--hash=sha256:ebaea975e03d3141d9d3a507df75c9b3ec90fa9d2ffd07567b3a978d9d790b26 \
|
||||
--hash=sha256:f0606c8bf2cdefea14a43530f7657cbbb7ecf1c4222512492ef4a4434a9501ec \
|
||||
--hash=sha256:f13c32a3abd6079a66d9526e18dad9b6d280384d49d7c54040cd57b6424041d9 \
|
||||
--hash=sha256:f7401aebd7f581d7f83a439d87d474999317ee099218e5ad25d125290990ba65 \
|
||||
--hash=sha256:fa4ecea169a355be7a3ade2c783e2ed12f0e40d2c5621cda8b3297faf7fbb9f5 \
|
||||
--hash=sha256:fbd139c8447d25dd750ab79ee274cc5e1fe80fc56340ab10b18a195e1b6eca3e \
|
||||
--hash=sha256:fdafc9cce40277e0f7a0feabce0ee50dd2fa1800f3b38015e51296b5e814048d \
|
||||
--hash=sha256:fe3cca2e4e8a592be0f269a1ca4835c25199d9f3ce815c8491048f785b0a0198 \
|
||||
--hash=sha256:ffd0c5368496f41b0944be820fcb7a838aa6e623d250b01acf2643939c3f99d7
|
||||
# via
|
||||
# -r .github/scripts/requirements_dev.in
|
||||
# pdf2image
|
||||
# weasyprint
|
||||
platformdirs==4.9.6 \
|
||||
--hash=sha256:3bfa75b0ad0db84096ae777218481852c0ebc6c727b3168c1b9e0118e458cf0a \
|
||||
--hash=sha256:e61adb1d5e5cb3441b4b7710bea7e4c12250ca49439228cc1021c00dcfac0917
|
||||
platformdirs==4.10.0 \
|
||||
--hash=sha256:31e761a6a0ca04faf7353ea759bdba55652be214725111e5aac52dfa29d4bef7 \
|
||||
--hash=sha256:fb516cdb12eb0d857d0cd85a7c57cea4d060bee4578d6cf5a14dfdf8cbf8784a
|
||||
# via
|
||||
# python-discovery
|
||||
# virtualenv
|
||||
@@ -381,9 +393,9 @@ pyphen==0.17.2 \
|
||||
--hash=sha256:3a07fb017cb2341e1d9ff31b8634efb1ae4dc4b130468c7c39dd3d32e7c3affd \
|
||||
--hash=sha256:f60647a9c9b30ec6c59910097af82bc5dd2d36576b918e44148d8b07ef3b4aa3
|
||||
# via weasyprint
|
||||
python-discovery==1.2.2 \
|
||||
--hash=sha256:876e9c57139eb757cb5878cbdd9ae5379e5d96266c99ef731119e04fffe533bb \
|
||||
--hash=sha256:e1ae95d9af875e78f15e19aed0c6137ab1bb49c200f21f5061786490c9585c7a
|
||||
python-discovery==1.4.4 \
|
||||
--hash=sha256:5cad33982d412c1f3ffb8f9ca4ea292c9680bca3942451d30b69c37fce53a4a3 \
|
||||
--hash=sha256:abebe9120b43453b68c908acfb1e72a19d1a959ed2cb620ad38fc57d08056dbe
|
||||
# via virtualenv
|
||||
pyyaml==6.0.3 \
|
||||
--hash=sha256:00c4bdeba853cc34e7dd471f16b4114f4162dc03e6b7afcc2128711f0eca823c \
|
||||
@@ -470,17 +482,17 @@ tinyhtml5==2.1.0 \
|
||||
--hash=sha256:60a50ec3d938a37e491efa01af895853060943dcebb5627de5b10d188b338a67 \
|
||||
--hash=sha256:6e11cfff38515834268daf89d5f85bbde0b6dd02e8d9e212d1385c2289b89f0a
|
||||
# via weasyprint
|
||||
unoserver==3.6 \
|
||||
--hash=sha256:25c360fa194396a89cb79b4edd2735f8e4f0fd8531e59db3952114585bd7df05 \
|
||||
--hash=sha256:e446bcb3638c51880f002aaeecab1cf74dfa9df81035f027f7ff2e081b6d7015
|
||||
unoserver==3.7 \
|
||||
--hash=sha256:b05f9578506ac7374ae1b314c3a79528636c542ac78220a9ce99110584ca424b \
|
||||
--hash=sha256:fc44e6808071c9d2957e705ecf1742cea8a582aa5d5cc23babf36bb332ec6e8e
|
||||
# via -r .github/scripts/requirements_dev.in
|
||||
virtualenv==21.2.4 \
|
||||
--hash=sha256:29d21e941795206138d0f22f4e45ff7050e5da6c6472299fb7103318763861ac \
|
||||
--hash=sha256:b294ef68192638004d72524ce7ef303e9d0cf5a44c95ce2e54a7500a6381cada
|
||||
virtualenv==21.6.1 \
|
||||
--hash=sha256:15f978b7cd329f24855ff4a0c4b4899cc7678589f49adbdcbbb4d3232e641128 \
|
||||
--hash=sha256:afe991df855715a2b2f60edfcc0107ef95a79fdfd8cb4cdaa71603d1c12e463b
|
||||
# via pre-commit
|
||||
weasyprint==68.1 \
|
||||
--hash=sha256:4dc3ba63c68bbbce3e9617cb2226251c372f5ee90a8a484503b1c099da9cf5be \
|
||||
--hash=sha256:d3b752049b453a5c95edb27ce78d69e9319af5a34f257fa0f4c738c701b4184e
|
||||
weasyprint==69.0 \
|
||||
--hash=sha256:475951cfd917014de6d4d005caff48c6aa867e7e42b80cd5b16a0484a1609ee6 \
|
||||
--hash=sha256:a7a32f39ca16bd82ef11de99c92ea4b5f14951c9033af035e451ce4f4ee0a88c
|
||||
# via -r .github/scripts/requirements_dev.in
|
||||
webencodings==0.5.1 \
|
||||
--hash=sha256:a0af1213f3c2226497a97e2b3aa01a7e4bee4f403f95be16fc9acd2947514a78 \
|
||||
@@ -489,27 +501,28 @@ webencodings==0.5.1 \
|
||||
# cssselect2
|
||||
# tinycss2
|
||||
# tinyhtml5
|
||||
zopfli==0.4.1 \
|
||||
--hash=sha256:02086247dd12fda929f9bfe8b3962b6bcdbfc8c82e99255aebcf367867cf0760 \
|
||||
--hash=sha256:07a5cdc5d1aaa6c288c5d9f5a5383042ba743641abf8e2fd898dcad622d8a38e \
|
||||
--hash=sha256:27823dc1161a4031d1c25925fd45d9868ec0cbc7692341830a7dcfa25063662c \
|
||||
--hash=sha256:2f992ac7d83cbddd889e1813ace576cbc91a05d5d7a0a21b366e2e5f492e7707 \
|
||||
--hash=sha256:4238d4d746d1095e29c9125490985e0c12ffd3654f54a24af551e2391e936d54 \
|
||||
--hash=sha256:5a4c22b6161f47f5bd34637dbaee6735abd287cd64e0d1ce28ef1871bf625f4b \
|
||||
--hash=sha256:84a31ba9edc921b1d3a4449929394a993888f32d70de3a3617800c428a947b9b \
|
||||
--hash=sha256:a899eca405662a23ae75054affa3517a060362eae1185d3d791c86a50153c4dd \
|
||||
--hash=sha256:a93c2ecafff372de6c0aa2212eff18a75f6c71a100372fee7b4b129cc0b6f9a7 \
|
||||
--hash=sha256:cb136a74d14a4ecfae29cb0fdecece58a6c115abc9a74c12bc6ac62e80f229d7 \
|
||||
--hash=sha256:d7bcee1b189d64ec33d1e05cfa1b6a1268c29329c382f6ca1bd6245b04925c57 \
|
||||
--hash=sha256:fdfb7ce9f5de37a5b2f75dd2642fd7717956ef2a72e0387302a36d382440db07
|
||||
zopfli==0.4.3 \
|
||||
--hash=sha256:0087c9a6f0c8a052be0f6d1a9bb71b6caffdd3e10201d6d6166e28d482cebe6d \
|
||||
--hash=sha256:47604eee5c6704bdf0e94d8391fe3b74ddb2abd84128fbcfdc3ee0fc265feaef \
|
||||
--hash=sha256:62248dbf8dbcbd588ee194b210e5be9fa80bce29641f55599d6d394bd2a9d8a3 \
|
||||
--hash=sha256:628c3e941752880b3491db8d44163d0aedb221944e22a17187ff7fc549b050f6 \
|
||||
--hash=sha256:769875152d0625c46707bcca57d4b2233fe653482067acd55fbf6ec525cb9bdc \
|
||||
--hash=sha256:7e9703ca6e7ef66c8d05e0826b6f558b680c9db8206f84f05a3ee93430a12e42 \
|
||||
--hash=sha256:7fa3c35193475290e3f007bbcdebdbae64ba2f012d75c632da0d727e1da50d5e \
|
||||
--hash=sha256:88f4fbe429aad72bc206275d81fab11a097e0f951a5848d1f51083c37ea73073 \
|
||||
--hash=sha256:921c2c9907f4364963848da5ad194b46d68865e07fdb975d04fd09bc42d47357 \
|
||||
--hash=sha256:d3a50f91a13cea9bafe025de8fd87a005eb26de02a4f0c193127ddbf23ac8ebe \
|
||||
--hash=sha256:d4f51dd1ab5312e837e2091284e0d9f1a138188f2e65812f9a5799dc02c45f94 \
|
||||
--hash=sha256:eb0c9c1d40a8cb1d58762d7e57290ccb753e0828c4d01be8acb59aae5d0ca206 \
|
||||
--hash=sha256:f2e0adcf7d36c6fd0dd36cc771ef7f0c5803a05666feafcd90d7170174a4148e
|
||||
# via fonttools
|
||||
|
||||
# The following packages are considered to be unsafe in a requirements file:
|
||||
pip==26.0.1 \
|
||||
--hash=sha256:bdb1b08f4274833d62c1aa29e20907365a2ceb950410df15fc9521bad440122b \
|
||||
--hash=sha256:c4037d8a277c89b320abe636d59f91e6d0922d08a05b60e85e53b296613346d8
|
||||
pip==26.1.2 \
|
||||
--hash=sha256:382ff9f685ee3bc25864f820aa50505825f10f5458ffff07e30a6d96e5715cab \
|
||||
--hash=sha256:f49cd134c61cf2fd75e0ce2676db03e4054504a5a4986d00f8299ae632dc4605
|
||||
# via -r .github/scripts/requirements_dev.in
|
||||
setuptools==82.0.1 \
|
||||
--hash=sha256:7d872682c5d01cfde07da7bccc7b65469d3dca203318515ada1de5eda35efbf9 \
|
||||
--hash=sha256:a59e362652f08dcd477c78bb6e7bd9d80a7995bc73ce773050228a348ce2e5bb
|
||||
setuptools==83.0.0 \
|
||||
--hash=sha256:025bccbbf0fa05b6192bc64ae1e7b16e001fd6d6d4d5de03c97b1c1ade523bef \
|
||||
--hash=sha256:29b23c360f22f414dc7336bb39178cc7bcbf6021ed2733cde173f09dba19abb3
|
||||
# via -r .github/scripts/requirements_dev.in
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
#
|
||||
# This file is autogenerated by pip-compile with Python 3.12
|
||||
# This file is autogenerated by pip-compile with Python 3.13
|
||||
# by the following command:
|
||||
#
|
||||
# pip-compile --generate-hashes --output-file='.github\scripts\requirements_sync_readme.txt' --strip-extras '.github\scripts\requirements_sync_readme.in'
|
||||
@@ -8,7 +8,7 @@ tomli-w==1.2.0 \
|
||||
--hash=sha256:188306098d013b691fcadc011abd66727d3c414c571bb01b1a174ba8c983cf90 \
|
||||
--hash=sha256:2dd14fac5a47c27be9cd4c976af5a12d87fb1f0b4512f81d69cce3b35ae25021
|
||||
# via -r .github/scripts/requirements_sync_readme.in
|
||||
tomlkit==0.14.0 \
|
||||
--hash=sha256:592064ed85b40fa213469f81ac584f67a4f2992509a7c3ea2d632208623a3680 \
|
||||
--hash=sha256:cf00efca415dbd57575befb1f6634c4f42d2d87dbba376128adb42c121b87064
|
||||
tomlkit==0.15.0 \
|
||||
--hash=sha256:4dbc8f0fc024412b57ced8757ac7461305126a648ff8c2c807fcb8e133a78738 \
|
||||
--hash=sha256:7d1a9ecba3086638211b13814ea79c90dd54dd11993564376f3aa92271f5c7a3
|
||||
# via -r .github/scripts/requirements_sync_readme.in
|
||||
|
||||
@@ -23,13 +23,9 @@ permissions:
|
||||
pull-requests: write
|
||||
|
||||
jobs:
|
||||
pick:
|
||||
uses: ./.github/workflows/_runner-pick.yml
|
||||
|
||||
check-pr:
|
||||
if: (github.event_name == 'pull_request' && github.event.action != 'closed') || github.event_name == 'workflow_dispatch'
|
||||
needs: pick
|
||||
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
|
||||
runs-on: ubuntu-latest
|
||||
outputs:
|
||||
should_deploy: ${{ steps.decide.outputs.should_deploy }}
|
||||
is_fork: ${{ steps.resolve.outputs.is_fork }}
|
||||
@@ -101,8 +97,8 @@ jobs:
|
||||
echo "allow_fork=${allow_fork:-false}" >> $GITHUB_OUTPUT
|
||||
|
||||
deploy-v2-pr:
|
||||
needs: [pick, check-pr]
|
||||
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
|
||||
needs: check-pr
|
||||
runs-on: ubuntu-latest
|
||||
if: needs.check-pr.outputs.should_deploy == 'true' && (needs.check-pr.outputs.is_fork == 'false' || needs.check-pr.outputs.allow_fork == 'true')
|
||||
# Concurrency control - only one deployment per PR at a time
|
||||
concurrency:
|
||||
@@ -112,10 +108,7 @@ jobs:
|
||||
contents: read
|
||||
issues: write
|
||||
pull-requests: write
|
||||
id-token: write
|
||||
env:
|
||||
USE_DEPOT: ${{ needs.pick.outputs.use_depot == 'true' }}
|
||||
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
|
||||
# Single source of truth for whether this preview embeds the admin portal:
|
||||
# drives the image build-arg and the deployment comment.
|
||||
BUILD_PORTAL: "true"
|
||||
@@ -190,12 +183,7 @@ jobs:
|
||||
token: ${{ secrets.GITHUB_TOKEN }}
|
||||
fetch-depth: 0 # Fetch full history for commit hash detection
|
||||
|
||||
- name: Set up Depot CLI
|
||||
if: env.USE_DEPOT == 'true'
|
||||
uses: depot/setup-action@15c09a5f77a0840ad4bce955686522a257853461 # v1.0.0
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
if: env.USE_DEPOT != 'true'
|
||||
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
|
||||
|
||||
- name: Get version number
|
||||
@@ -240,22 +228,8 @@ jobs:
|
||||
echo "Image needs to be built"
|
||||
fi
|
||||
|
||||
- name: Build and push V2 image (Depot)
|
||||
if: env.USE_DEPOT == 'true' && steps.check-image.outputs.exists == 'false'
|
||||
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.16.0
|
||||
with:
|
||||
project: ${{ vars.DEPOT_PROJECT_ID }}
|
||||
context: .
|
||||
file: ./docker/embedded/Dockerfile
|
||||
push: true
|
||||
tags: ${{ secrets.DOCKER_HUB_USERNAME }}/test:v2-${{ steps.commit-hash.outputs.app_short }}
|
||||
build-args: |
|
||||
VERSION_TAG=v2-alpha
|
||||
BUILD_PORTAL=${{ env.BUILD_PORTAL }}
|
||||
platforms: linux/amd64
|
||||
|
||||
- name: Build and push V2 image (Docker fork fallback)
|
||||
if: env.USE_DEPOT != 'true' && steps.check-image.outputs.exists == 'false'
|
||||
- name: Build and push V2 image
|
||||
if: steps.check-image.outputs.exists == 'false'
|
||||
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
|
||||
with:
|
||||
context: .
|
||||
@@ -474,8 +448,7 @@ jobs:
|
||||
|
||||
cleanup-v2-deployment:
|
||||
if: github.event.action == 'closed'
|
||||
needs: pick
|
||||
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: read
|
||||
issues: write
|
||||
|
||||
@@ -34,12 +34,8 @@ permissions:
|
||||
pull-requests: read
|
||||
|
||||
jobs:
|
||||
pick:
|
||||
uses: ./.github/workflows/_runner-pick.yml
|
||||
|
||||
check-comment:
|
||||
needs: pick
|
||||
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
issues: write
|
||||
if: |
|
||||
@@ -179,15 +175,11 @@ jobs:
|
||||
}
|
||||
|
||||
deploy-pr:
|
||||
needs: [pick, check-comment]
|
||||
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
|
||||
needs: check-comment
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
issues: write
|
||||
pull-requests: write
|
||||
id-token: write
|
||||
env:
|
||||
USE_DEPOT: ${{ needs.pick.outputs.use_depot == 'true' }}
|
||||
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
|
||||
|
||||
steps:
|
||||
- name: Harden Runner
|
||||
@@ -220,9 +212,9 @@ jobs:
|
||||
distribution: "temurin"
|
||||
|
||||
- name: Setup Gradle
|
||||
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
|
||||
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
|
||||
with:
|
||||
gradle-version: 9.6.0
|
||||
gradle-version: 9.6.1
|
||||
|
||||
- name: Install Task
|
||||
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
|
||||
@@ -240,12 +232,7 @@ jobs:
|
||||
MAVEN_PUBLIC_URL: ${{ secrets.MAVEN_PUBLIC_URL }}
|
||||
STIRLING_PDF_DESKTOP_UI: false
|
||||
|
||||
- name: Set up Depot CLI
|
||||
if: env.USE_DEPOT == 'true'
|
||||
uses: depot/setup-action@15c09a5f77a0840ad4bce955686522a257853461 # v1.0.0
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
if: env.USE_DEPOT != 'true'
|
||||
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
|
||||
|
||||
- name: Login to Docker Hub
|
||||
@@ -254,22 +241,7 @@ jobs:
|
||||
username: ${{ secrets.DOCKER_HUB_USERNAME }}
|
||||
password: ${{ secrets.DOCKER_HUB_API }}
|
||||
|
||||
- name: Build and push PR-specific image (Depot)
|
||||
if: env.USE_DEPOT == 'true'
|
||||
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.16.0
|
||||
with:
|
||||
project: ${{ vars.DEPOT_PROJECT_ID }}
|
||||
context: .
|
||||
file: ./docker/embedded/Dockerfile
|
||||
push: true
|
||||
tags: ${{ secrets.DOCKER_HUB_USERNAME }}/test:pr-${{ needs.check-comment.outputs.pr_number }}
|
||||
build-args: |
|
||||
VERSION_TAG=alpha
|
||||
PROTOTYPES_BUILD=${{ needs.check-comment.outputs.enable_prototypes }}
|
||||
platforms: linux/amd64
|
||||
|
||||
- name: Build and push PR-specific image (Docker fork fallback)
|
||||
if: env.USE_DEPOT != 'true'
|
||||
- name: Build and push PR-specific image
|
||||
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
|
||||
with:
|
||||
context: .
|
||||
@@ -283,19 +255,8 @@ jobs:
|
||||
PROTOTYPES_BUILD=${{ needs.check-comment.outputs.enable_prototypes }}
|
||||
platforms: linux/amd64
|
||||
|
||||
- name: Build and push engine image (Depot)
|
||||
if: env.USE_DEPOT == 'true' && needs.check-comment.outputs.enable_prototypes == 'true'
|
||||
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.16.0
|
||||
with:
|
||||
project: ${{ vars.DEPOT_PROJECT_ID }}
|
||||
context: ./engine
|
||||
file: ./engine/Dockerfile
|
||||
push: true
|
||||
tags: ${{ secrets.DOCKER_HUB_USERNAME }}/test:engine-pr-${{ needs.check-comment.outputs.pr_number }}
|
||||
platforms: linux/amd64
|
||||
|
||||
- name: Build and push engine image (Docker fork fallback)
|
||||
if: env.USE_DEPOT != 'true' && needs.check-comment.outputs.enable_prototypes == 'true'
|
||||
- name: Build and push engine image
|
||||
if: needs.check-comment.outputs.enable_prototypes == 'true'
|
||||
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
|
||||
with:
|
||||
context: ./engine
|
||||
@@ -510,8 +471,7 @@ jobs:
|
||||
|
||||
handle-label-commands:
|
||||
if: ${{ github.event.issue.pull_request != null }}
|
||||
needs: pick
|
||||
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Harden Runner
|
||||
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
|
||||
|
||||
@@ -2,13 +2,8 @@ name: _runner-pick
|
||||
|
||||
# Tiny reusable workflow that classifies the trigger as either a "fork PR
|
||||
# from an untrusted contributor" or a "trusted commit" so downstream jobs
|
||||
# can pick a runner class without each one duplicating the 200-char gate
|
||||
# expression in their own `runs-on:`.
|
||||
#
|
||||
# It also owns the single Depot kill-switch (use_depot). Depot is currently
|
||||
# disabled repo-wide; downstream jobs gate their Depot runner/build usage on
|
||||
# use_depot so nothing has to be deleted to turn Depot off. Flip DEPOT_ENABLED
|
||||
# in the decide step to switch Depot back on.
|
||||
# can trust-gate (skip secret-dependent jobs on forks) without each one
|
||||
# duplicating the gate expression.
|
||||
#
|
||||
# Caller pattern:
|
||||
#
|
||||
@@ -18,15 +13,12 @@ name: _runner-pick
|
||||
#
|
||||
# real-work:
|
||||
# needs: pick
|
||||
# runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-8' || 'ubuntu-latest' }}
|
||||
# if: needs.pick.outputs.is_fork != 'true'
|
||||
# steps: [...]
|
||||
#
|
||||
# Outputs:
|
||||
# is_fork: "true" when the trigger is a pull_request from a fork or an
|
||||
# untrusted author_association, "false" otherwise. Use this for
|
||||
# trust gating (skipping secret-dependent jobs on forks).
|
||||
# use_depot: "true" when downstream jobs should use Depot runners/builders.
|
||||
# Currently forced "false" (Depot disabled repo-wide).
|
||||
# untrusted author_association, "false" otherwise.
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
@@ -34,9 +26,6 @@ on:
|
||||
is_fork:
|
||||
description: '"true" if the trigger is an untrusted fork PR.'
|
||||
value: ${{ jobs.pick.outputs.is_fork }}
|
||||
use_depot:
|
||||
description: '"true" when downstream jobs should use Depot. Currently forced off.'
|
||||
value: ${{ jobs.pick.outputs.use_depot }}
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
@@ -47,7 +36,6 @@ jobs:
|
||||
timeout-minutes: 1
|
||||
outputs:
|
||||
is_fork: ${{ steps.decide.outputs.is_fork }}
|
||||
use_depot: ${{ steps.decide.outputs.use_depot }}
|
||||
steps:
|
||||
- name: Harden the runner (Audit all outbound calls)
|
||||
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
|
||||
@@ -63,12 +51,6 @@ jobs:
|
||||
run: |
|
||||
set -eu
|
||||
|
||||
# Depot kill-switch. Depot is disabled repo-wide: no job uses Depot
|
||||
# runners or the Depot build actions while this is false. All the
|
||||
# Depot wiring is left in place - set DEPOT_ENABLED=true to switch it
|
||||
# back on (it then activates on trusted, non-fork triggers as before).
|
||||
DEPOT_ENABLED=false
|
||||
|
||||
if [ -z "${PR_NUMBER:-}" ]; then
|
||||
# Not a pull_request event at all (push, schedule, workflow_dispatch,
|
||||
# workflow_call from a non-PR trigger) -> trusted by default.
|
||||
@@ -82,13 +64,4 @@ jobs:
|
||||
esac
|
||||
fi
|
||||
|
||||
# Depot only ever ran on trusted triggers, so gate it on both the
|
||||
# kill-switch and is_fork.
|
||||
if [ "${DEPOT_ENABLED}" = "true" ] && [ "${is_fork}" = "false" ]; then
|
||||
use_depot=true
|
||||
else
|
||||
use_depot=false
|
||||
fi
|
||||
|
||||
echo "is_fork=${is_fork}" >> "$GITHUB_OUTPUT"
|
||||
echo "use_depot=${use_depot}" >> "$GITHUB_OUTPUT"
|
||||
|
||||
@@ -18,8 +18,6 @@ jobs:
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: write
|
||||
env:
|
||||
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
|
||||
steps:
|
||||
- name: Harden the runner (Audit all outbound calls)
|
||||
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
|
||||
@@ -33,6 +31,7 @@ jobs:
|
||||
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
|
||||
with:
|
||||
enable-cache: true
|
||||
cache-suffix: ai-engine
|
||||
|
||||
- name: Install Task
|
||||
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
|
||||
|
||||
@@ -19,14 +19,8 @@ permissions:
|
||||
pull-requests: write
|
||||
|
||||
jobs:
|
||||
pick:
|
||||
uses: ./.github/workflows/_runner-pick.yml
|
||||
|
||||
build:
|
||||
needs: pick
|
||||
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-8' || 'ubuntu-latest' }}
|
||||
env:
|
||||
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
@@ -56,9 +50,9 @@ jobs:
|
||||
key: gradle-deps-${{ runner.os }}-jdk-${{ matrix.jdk-version }}-${{ hashFiles('**/gradle/wrapper/gradle-wrapper.properties', '**/*.gradle', '**/*.gradle.kts', 'settings.gradle', 'settings.gradle.kts', 'gradle/libs.versions.toml') }}
|
||||
|
||||
- name: Setup Gradle
|
||||
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
|
||||
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
|
||||
with:
|
||||
gradle-version: 9.6.0
|
||||
gradle-version: 9.6.1
|
||||
cache-disabled: true
|
||||
|
||||
- name: Install Task
|
||||
@@ -247,7 +241,7 @@ jobs:
|
||||
# so skip it for merge_group runs and workflow_dispatch.
|
||||
if: github.event_name == 'pull_request'
|
||||
id: jacoco
|
||||
uses: madrapps/jacoco-report@50d3aff4548aa991e6753342d9ba291084e63848 # v1.7.2
|
||||
uses: madrapps/jacoco-report@e51ce1f46f7f8b5331593f935e59cbaf44b84920 # v1.8.0
|
||||
with:
|
||||
paths: |
|
||||
${{ github.workspace }}/**/build/reports/jacoco/test/jacocoTestReport.xml
|
||||
|
||||
@@ -15,23 +15,11 @@ name: Enterprise E2E (Playwright)
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
depot_cores:
|
||||
description: "Depot runner vCPU count (used in runs-on). Override for benchmarking."
|
||||
required: false
|
||||
type: string
|
||||
default: "8"
|
||||
push:
|
||||
branches: ["main"]
|
||||
schedule:
|
||||
- cron: "0 4 * * *"
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
depot_cores:
|
||||
description: "Depot runner vCPU count (used in runs-on). Override for benchmarking."
|
||||
required: false
|
||||
type: string
|
||||
default: "8"
|
||||
|
||||
# No `concurrency:` block here on purpose. When this workflow is called via
|
||||
# workflow_call from build.yml, ${{ github.workflow }}/event_name/pr_number
|
||||
@@ -50,17 +38,16 @@ jobs:
|
||||
|
||||
playwright-e2e-enterprise:
|
||||
needs: pick
|
||||
# Skip on fork PRs / untrusted authors: they have no PREMIUM_KEY_ENTERPRISE
|
||||
# (nor DEPOT_TOKEN), so the suite can't boot premium and would fail. See the
|
||||
# header comment. GitHub reports the skipped reusable workflow as success.
|
||||
# Skip on fork PRs / untrusted authors: they have no PREMIUM_KEY_ENTERPRISE,
|
||||
# so the suite can't boot premium and would fail. See the header comment.
|
||||
# GitHub reports the skipped reusable workflow as success.
|
||||
if: needs.pick.outputs.is_fork != 'true'
|
||||
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && format('depot-ubuntu-24.04-{0}', inputs.depot_cores || '8') || 'ubuntu-latest' }}
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 45
|
||||
env:
|
||||
PREMIUM_KEY: ${{ secrets.PREMIUM_KEY_ENTERPRISE }}
|
||||
PREMIUM_ENABLED: "true"
|
||||
SYSTEM_ENABLEANALYTICS: "false"
|
||||
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
|
||||
steps:
|
||||
- name: Harden Runner
|
||||
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
|
||||
|
||||
@@ -99,6 +99,17 @@ jobs:
|
||||
uses: ./.github/workflows/frontend-validation.yml
|
||||
secrets: inherit
|
||||
|
||||
# Advisory: deliberately NOT in all-checks-passed. It reports on the stories a
|
||||
# branch touches so a regression is visible in review, but a browser scan is
|
||||
# too new here to block merges on. Promote it once its pass/fail proves stable.
|
||||
frontend-a11y:
|
||||
if: needs.files-changed.outputs.frontend == 'true'
|
||||
needs: [files-changed]
|
||||
permissions:
|
||||
contents: read
|
||||
uses: ./.github/workflows/frontend-a11y.yml
|
||||
secrets: inherit
|
||||
|
||||
playwright-e2e:
|
||||
if: needs.files-changed.outputs.frontend == 'true'
|
||||
needs: [files-changed]
|
||||
@@ -149,7 +160,6 @@ jobs:
|
||||
permissions:
|
||||
contents: read
|
||||
packages: read
|
||||
id-token: write
|
||||
uses: ./.github/workflows/test-build-docker.yml
|
||||
secrets: inherit
|
||||
with:
|
||||
@@ -164,13 +174,12 @@ jobs:
|
||||
pull-requests: write
|
||||
uses: ./.github/workflows/tauri-build.yml
|
||||
secrets: inherit
|
||||
# PR smoke build: Linux only (fastest + cheapest to compile), unsigned,
|
||||
# deb-only, no AppImage. The full signed multi-OS matrix runs on release;
|
||||
# PR smoke build: macOS + Windows (the platforms our developers use).
|
||||
# The full signed multi-OS matrix runs on release;
|
||||
# nightly still warms the Rust cache with all-OS defaults.
|
||||
with:
|
||||
platform: linux
|
||||
platform: windows-macos
|
||||
sign: false
|
||||
minimal: true
|
||||
|
||||
ai-engine:
|
||||
if: needs.files-changed.outputs.engine == 'true'
|
||||
|
||||
@@ -21,8 +21,6 @@ jobs:
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: write
|
||||
env:
|
||||
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
|
||||
steps:
|
||||
- name: Harden the runner (Audit all outbound calls)
|
||||
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
|
||||
@@ -36,6 +34,7 @@ jobs:
|
||||
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
|
||||
with:
|
||||
enable-cache: true
|
||||
cache-suffix: generated-models
|
||||
|
||||
- name: Set up JDK 25
|
||||
uses: actions/setup-java@be666c2fcd27ec809703dec50e508c2fdc7f6654 # v5.2.0
|
||||
@@ -44,7 +43,7 @@ jobs:
|
||||
distribution: "temurin"
|
||||
|
||||
- name: Setup Gradle
|
||||
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
|
||||
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
|
||||
with:
|
||||
gradle-version: 9.6.0
|
||||
|
||||
|
||||
@@ -11,8 +11,6 @@ permissions:
|
||||
jobs:
|
||||
check-licence:
|
||||
runs-on: ubuntu-latest
|
||||
env:
|
||||
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
|
||||
steps:
|
||||
- name: Harden Runner
|
||||
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
|
||||
@@ -38,9 +36,9 @@ jobs:
|
||||
key: gradle-deps-${{ runner.os }}-jdk-25-${{ hashFiles('**/gradle/wrapper/gradle-wrapper.properties', '**/*.gradle', '**/*.gradle.kts', 'settings.gradle', 'settings.gradle.kts', 'gradle/libs.versions.toml') }}
|
||||
|
||||
- name: Setup Gradle
|
||||
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
|
||||
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
|
||||
with:
|
||||
gradle-version: 9.6.0
|
||||
gradle-version: 9.6.1
|
||||
cache-disabled: true
|
||||
|
||||
- name: Install Task
|
||||
|
||||
@@ -10,14 +10,8 @@ permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
pick:
|
||||
uses: ./.github/workflows/_runner-pick.yml
|
||||
|
||||
check-generate-openapi-docs:
|
||||
needs: pick
|
||||
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
|
||||
env:
|
||||
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Harden Runner
|
||||
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
|
||||
@@ -43,9 +37,9 @@ jobs:
|
||||
key: gradle-deps-${{ runner.os }}-jdk-25-${{ hashFiles('**/gradle/wrapper/gradle-wrapper.properties', '**/*.gradle', '**/*.gradle.kts', 'settings.gradle', 'settings.gradle.kts', 'gradle/libs.versions.toml') }}
|
||||
|
||||
- name: Setup Gradle
|
||||
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
|
||||
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
|
||||
with:
|
||||
gradle-version: 9.6.0
|
||||
gradle-version: 9.6.1
|
||||
cache-disabled: true
|
||||
|
||||
- name: Install Task
|
||||
|
||||
@@ -196,7 +196,7 @@ jobs:
|
||||
core.exportVariable("REFERENCE_FILE", referenceFilePath);
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
|
||||
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
|
||||
@@ -29,12 +29,8 @@ permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
pick:
|
||||
uses: ./.github/workflows/_runner-pick.yml
|
||||
|
||||
aggregate:
|
||||
needs: pick
|
||||
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 15
|
||||
steps:
|
||||
- name: Harden Runner
|
||||
@@ -60,13 +56,13 @@ jobs:
|
||||
key: gradle-deps-${{ runner.os }}-jdk-25-${{ hashFiles('**/gradle/wrapper/gradle-wrapper.properties', '**/*.gradle', '**/*.gradle.kts', 'settings.gradle', 'settings.gradle.kts', 'gradle/libs.versions.toml') }}
|
||||
|
||||
- name: Setup Gradle
|
||||
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
|
||||
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
|
||||
with:
|
||||
gradle-version: 9.6.0
|
||||
gradle-version: 9.6.1
|
||||
cache-disabled: true
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
|
||||
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
|
||||
@@ -12,15 +12,9 @@ permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
pick:
|
||||
uses: ./.github/workflows/_runner-pick.yml
|
||||
|
||||
migration-test:
|
||||
needs: pick
|
||||
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-8' || 'ubuntu-latest' }}
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 30
|
||||
env:
|
||||
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
|
||||
steps:
|
||||
- name: Harden Runner
|
||||
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
|
||||
@@ -46,9 +40,9 @@ jobs:
|
||||
key: gradle-deps-${{ runner.os }}-jdk-25-${{ hashFiles('**/gradle/wrapper/gradle-wrapper.properties', '**/*.gradle', '**/*.gradle.kts', 'settings.gradle', 'settings.gradle.kts', 'gradle/libs.versions.toml') }}
|
||||
|
||||
- name: Setup Gradle
|
||||
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
|
||||
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
|
||||
with:
|
||||
gradle-version: 9.6.0
|
||||
gradle-version: 9.6.1
|
||||
cache-disabled: true
|
||||
|
||||
# No `-PnoSpotless` here yet because the upstream cache layer matches the
|
||||
|
||||
@@ -10,21 +10,11 @@ permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
pick:
|
||||
uses: ./.github/workflows/_runner-pick.yml
|
||||
|
||||
deploy-v2-on-push:
|
||||
needs: pick
|
||||
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
|
||||
runs-on: ubuntu-latest
|
||||
concurrency:
|
||||
group: deploy-v2-push-V2
|
||||
cancel-in-progress: true
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
env:
|
||||
USE_DEPOT: ${{ needs.pick.outputs.use_depot == 'true' }}
|
||||
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
|
||||
|
||||
steps:
|
||||
- name: Harden Runner
|
||||
@@ -35,12 +25,7 @@ jobs:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
|
||||
- name: Set up Depot CLI
|
||||
if: env.USE_DEPOT == 'true'
|
||||
uses: depot/setup-action@15c09a5f77a0840ad4bce955686522a257853461 # v1.0.0
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
if: env.USE_DEPOT != 'true'
|
||||
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
|
||||
|
||||
- name: Get commit hashes for frontend and backend
|
||||
@@ -105,22 +90,8 @@ jobs:
|
||||
username: ${{ secrets.DOCKER_HUB_USERNAME }}
|
||||
password: ${{ secrets.DOCKER_HUB_API }}
|
||||
|
||||
- name: Build and push frontend image (Depot)
|
||||
if: env.USE_DEPOT == 'true' && steps.check-frontend.outputs.exists == 'false'
|
||||
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.16.0
|
||||
with:
|
||||
project: ${{ vars.DEPOT_PROJECT_ID }}
|
||||
context: .
|
||||
file: ./docker/frontend/Dockerfile
|
||||
push: true
|
||||
tags: |
|
||||
${{ secrets.DOCKER_HUB_USERNAME }}/test:v2-frontend-${{ steps.commit-hashes.outputs.frontend_short }}
|
||||
${{ secrets.DOCKER_HUB_USERNAME }}/test:v2-frontend-latest
|
||||
build-args: VERSION_TAG=v2-alpha
|
||||
platforms: linux/amd64
|
||||
|
||||
- name: Build and push frontend image (Docker fork fallback)
|
||||
if: env.USE_DEPOT != 'true' && steps.check-frontend.outputs.exists == 'false'
|
||||
- name: Build and push frontend image
|
||||
if: steps.check-frontend.outputs.exists == 'false'
|
||||
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
|
||||
with:
|
||||
context: .
|
||||
@@ -134,22 +105,8 @@ jobs:
|
||||
build-args: VERSION_TAG=v2-alpha
|
||||
platforms: linux/amd64
|
||||
|
||||
- name: Build and push backend image (Depot)
|
||||
if: env.USE_DEPOT == 'true' && steps.check-backend.outputs.exists == 'false'
|
||||
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.16.0
|
||||
with:
|
||||
project: ${{ vars.DEPOT_PROJECT_ID }}
|
||||
context: .
|
||||
file: ./docker/backend/Dockerfile
|
||||
push: true
|
||||
tags: |
|
||||
${{ secrets.DOCKER_HUB_USERNAME }}/test:v2-backend-${{ steps.commit-hashes.outputs.backend_short }}
|
||||
${{ secrets.DOCKER_HUB_USERNAME }}/test:v2-backend-latest
|
||||
build-args: VERSION_TAG=v2-alpha
|
||||
platforms: linux/amd64
|
||||
|
||||
- name: Build and push backend image (Docker fork fallback)
|
||||
if: env.USE_DEPOT != 'true' && steps.check-backend.outputs.exists == 'false'
|
||||
- name: Build and push backend image
|
||||
if: steps.check-backend.outputs.exists == 'false'
|
||||
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
|
||||
with:
|
||||
context: .
|
||||
|
||||
@@ -11,28 +11,17 @@ on:
|
||||
required: false
|
||||
type: string
|
||||
default: "false"
|
||||
depot_cores:
|
||||
description: "Depot runner vCPU count (used in runs-on). Override for benchmarking. Tuned to 4 because bench showed 16 was within noise of 4."
|
||||
required: false
|
||||
type: string
|
||||
default: "4"
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
pick:
|
||||
uses: ./.github/workflows/_runner-pick.yml
|
||||
|
||||
docker-compose-tests:
|
||||
needs: pick
|
||||
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && format('depot-ubuntu-24.04-{0}', inputs.depot_cores || '4') || 'ubuntu-latest' }}
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
actions: write
|
||||
contents: read
|
||||
checks: write
|
||||
env:
|
||||
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
|
||||
|
||||
steps:
|
||||
- name: Harden Runner
|
||||
@@ -59,9 +48,9 @@ jobs:
|
||||
key: gradle-deps-${{ runner.os }}-jdk-25-${{ hashFiles('**/gradle/wrapper/gradle-wrapper.properties', '**/*.gradle', '**/*.gradle.kts', 'settings.gradle', 'settings.gradle.kts', 'gradle/libs.versions.toml') }}
|
||||
|
||||
- name: Setup Gradle
|
||||
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
|
||||
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
|
||||
with:
|
||||
gradle-version: 9.6.0
|
||||
gradle-version: 9.6.1
|
||||
cache-disabled: true
|
||||
|
||||
# When the PR changes the base image, test.sh builds it locally
|
||||
@@ -85,7 +74,7 @@ jobs:
|
||||
sudo chmod +x /usr/local/bin/docker-compose
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
|
||||
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
|
||||
with:
|
||||
python-version: "3.12"
|
||||
cache: "pip" # caching pip dependencies
|
||||
|
||||
@@ -5,23 +5,13 @@ name: Playwright E2E (live backend)
|
||||
# server.
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
depot_cores:
|
||||
description: "Depot runner vCPU count (used in runs-on). Override for benchmarking."
|
||||
required: false
|
||||
type: string
|
||||
default: "8"
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
pick:
|
||||
uses: ./.github/workflows/_runner-pick.yml
|
||||
|
||||
playwright-e2e-live:
|
||||
needs: pick
|
||||
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && format('depot-ubuntu-24.04-{0}', inputs.depot_cores || '8') || 'ubuntu-latest' }}
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 30
|
||||
steps:
|
||||
- name: Harden Runner
|
||||
@@ -94,7 +84,7 @@ jobs:
|
||||
fi
|
||||
- name: Set up Python for coverage summary
|
||||
if: always() && steps.live-coverage.outputs.report == 'true'
|
||||
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
|
||||
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
|
||||
with:
|
||||
python-version: "3.12"
|
||||
- name: Install defusedxml for coverage summary
|
||||
@@ -134,7 +124,7 @@ jobs:
|
||||
# a summary even on backend failure, as long as some Playwright
|
||||
# tests ran far enough to dump V8 coverage.
|
||||
if: always()
|
||||
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
|
||||
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
|
||||
@@ -5,23 +5,13 @@ name: Playwright E2E (stubbed)
|
||||
# mocks API responses in the browser.
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
depot_cores:
|
||||
description: "Depot runner vCPU count (used in runs-on). Override for benchmarking. Tuned to 8 to match the other playwright workflows; bench showed flat scaling above 8."
|
||||
required: false
|
||||
type: string
|
||||
default: "8"
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
pick:
|
||||
uses: ./.github/workflows/_runner-pick.yml
|
||||
|
||||
playwright-e2e:
|
||||
needs: pick
|
||||
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && format('depot-ubuntu-24.04-{0}', inputs.depot_cores || '8') || 'ubuntu-latest' }}
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Harden Runner
|
||||
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
|
||||
|
||||
@@ -0,0 +1,60 @@
|
||||
name: Frontend a11y regression gate
|
||||
|
||||
# Reusable workflow called from build.yml when frontend sources change.
|
||||
#
|
||||
# Scans the stories this branch touches in real Chromium and runs axe against
|
||||
# each. Existing violations are grandfathered in .storybook/a11y-baseline.json;
|
||||
# the check fails on a NEW violation — a story breaking a rule it wasn't already
|
||||
# breaking — or on a story that fails to render at all.
|
||||
#
|
||||
# Only changed stories, because a full sweep is ~30 minutes: far too slow to sit
|
||||
# in front of every merge. The whole suite is scanned nightly instead
|
||||
# (nightly.yml), which catches anything a branch didn't touch.
|
||||
#
|
||||
# Advisory for now: this is not in build.yml's all-checks-passed list, so a
|
||||
# failure reports without blocking. Promote it once a few weeks of runs show the
|
||||
# pass/fail is stable.
|
||||
on:
|
||||
workflow_call:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
frontend-a11y:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 25
|
||||
steps:
|
||||
- name: Harden Runner
|
||||
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
|
||||
with:
|
||||
egress-policy: audit
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
# Need the base branch too, to diff against it.
|
||||
fetch-depth: 0
|
||||
- name: Set up Node.js
|
||||
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
|
||||
with:
|
||||
node-version: "22"
|
||||
cache: "npm"
|
||||
cache-dependency-path: frontend/package-lock.json
|
||||
- name: Install Task
|
||||
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
|
||||
- name: a11y gate (changed stories)
|
||||
run: task frontend:storybook:a11y:changed -- origin/${{ github.base_ref || 'main' }}
|
||||
- name: Upload scan reports
|
||||
# The reports carry the offending selector and help text for each
|
||||
# violation; without them a red run can only be understood by
|
||||
# reproducing the whole scan locally.
|
||||
if: always()
|
||||
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
|
||||
with:
|
||||
name: a11y-scan-${{ github.run_id }}
|
||||
path: frontend/.a11y-scan/
|
||||
retention-days: 7
|
||||
if-no-files-found: ignore
|
||||
# The reports live in a dot-directory, which upload-artifact treats as
|
||||
# hidden and silently skips by default.
|
||||
include-hidden-files: true
|
||||
@@ -19,13 +19,9 @@ permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
pick:
|
||||
uses: ./.github/workflows/_runner-pick.yml
|
||||
|
||||
files-changed:
|
||||
name: detect what files changed
|
||||
needs: pick
|
||||
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 3
|
||||
outputs:
|
||||
licenses-frontend: ${{ steps.changes.outputs.licenses-frontend }}
|
||||
@@ -48,8 +44,8 @@ jobs:
|
||||
generate-frontend-license-report:
|
||||
if: needs.files-changed.outputs.licenses-frontend == 'true'
|
||||
name: Generate Frontend License Report
|
||||
needs: [pick, files-changed]
|
||||
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
|
||||
needs: files-changed
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
@@ -299,7 +295,10 @@ jobs:
|
||||
base: main
|
||||
title: "Update Frontend 3rd Party Licenses"
|
||||
body: ${{ env.PR_BODY }}
|
||||
labels: Licenses,github-actions,frontend
|
||||
labels: |
|
||||
Licenses
|
||||
github-actions
|
||||
Front End
|
||||
draft: false
|
||||
delete-branch: true
|
||||
sign-commits: true
|
||||
@@ -318,15 +317,13 @@ jobs:
|
||||
|
||||
generate-backend-license-report:
|
||||
if: needs.files-changed.outputs.licenses-backend == 'true'
|
||||
needs: [pick, files-changed]
|
||||
needs: files-changed
|
||||
name: Generate Backend License Report
|
||||
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
repository-projects: write # Required for enabling automerge
|
||||
env:
|
||||
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
|
||||
steps:
|
||||
- name: Harden Runner
|
||||
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
|
||||
@@ -354,9 +351,9 @@ jobs:
|
||||
distribution: "temurin"
|
||||
|
||||
- name: Setup Gradle
|
||||
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
|
||||
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
|
||||
with:
|
||||
gradle-version: 9.6.0
|
||||
gradle-version: 9.6.1
|
||||
|
||||
- name: Install Task
|
||||
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
|
||||
@@ -520,7 +517,10 @@ jobs:
|
||||
base: main
|
||||
title: "Update Backend 3rd Party Licenses"
|
||||
body: ${{ env.PR_BODY }}
|
||||
labels: Licenses,github-actions,backend
|
||||
labels: |
|
||||
Licenses
|
||||
github-actions
|
||||
Back End
|
||||
delete-branch: true
|
||||
sign-commits: true
|
||||
|
||||
|
||||
@@ -11,12 +11,8 @@ permissions:
|
||||
pull-requests: write
|
||||
|
||||
jobs:
|
||||
pick:
|
||||
uses: ./.github/workflows/_runner-pick.yml
|
||||
|
||||
frontend-validation:
|
||||
needs: pick
|
||||
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Harden Runner
|
||||
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
|
||||
@@ -121,7 +117,7 @@ jobs:
|
||||
run: task frontend:test:coverage
|
||||
- name: Set up Python for coverage summary
|
||||
if: always()
|
||||
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
|
||||
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
|
||||
with:
|
||||
python-version: "3.12"
|
||||
- name: Install defusedxml for coverage summary
|
||||
|
||||
@@ -36,13 +36,9 @@ permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
pick:
|
||||
uses: ./.github/workflows/_runner-pick.yml
|
||||
|
||||
determine-matrix:
|
||||
if: ${{ vars.CI_PROFILE != 'lite' }}
|
||||
needs: pick
|
||||
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
|
||||
runs-on: ubuntu-latest
|
||||
outputs:
|
||||
matrix: ${{ steps.set-matrix.outputs.matrix }}
|
||||
version: ${{ steps.versionNumber.outputs.versionNumber }}
|
||||
@@ -71,9 +67,9 @@ jobs:
|
||||
gradle-${{ runner.os }}-
|
||||
|
||||
- name: Setup Gradle
|
||||
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
|
||||
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
|
||||
with:
|
||||
gradle-version: 9.6.0
|
||||
gradle-version: 9.6.1
|
||||
|
||||
- name: Install Task
|
||||
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
|
||||
@@ -112,10 +108,8 @@ jobs:
|
||||
fi
|
||||
|
||||
build-jars:
|
||||
needs: [pick, determine-matrix]
|
||||
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
|
||||
env:
|
||||
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
|
||||
needs: determine-matrix
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
matrix:
|
||||
variant:
|
||||
@@ -146,9 +140,9 @@ jobs:
|
||||
distribution: "temurin"
|
||||
|
||||
- name: Setup Gradle
|
||||
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
|
||||
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
|
||||
with:
|
||||
gradle-version: 9.6.0
|
||||
gradle-version: 9.6.1
|
||||
|
||||
- name: Setup Node.js
|
||||
if: matrix.variant.build_frontend == true
|
||||
@@ -195,7 +189,6 @@ jobs:
|
||||
SM_API_KEY: ${{ secrets.SM_API_KEY }}
|
||||
WINDOWS_CERTIFICATE: ${{ secrets.WINDOWS_CERTIFICATE }}
|
||||
RELEASE_GPG_PRIVATE_KEY: ${{ secrets.RELEASE_GPG_PRIVATE_KEY }}
|
||||
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
|
||||
steps:
|
||||
- name: Harden Runner
|
||||
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
|
||||
@@ -250,9 +243,9 @@ jobs:
|
||||
distribution: "temurin"
|
||||
|
||||
- name: Setup Gradle
|
||||
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
|
||||
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
|
||||
with:
|
||||
gradle-version: 9.6.0
|
||||
gradle-version: 9.6.1
|
||||
|
||||
- name: Install Task
|
||||
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
|
||||
@@ -638,8 +631,8 @@ jobs:
|
||||
retention-days: 1
|
||||
|
||||
collect-and-release:
|
||||
needs: [pick, determine-matrix, build, build-jars]
|
||||
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
|
||||
needs: [determine-matrix, build, build-jars]
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: write
|
||||
steps:
|
||||
|
||||
@@ -13,13 +13,9 @@ permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
pick:
|
||||
uses: ./.github/workflows/_runner-pick.yml
|
||||
|
||||
playwright-all-browsers:
|
||||
name: Playwright (chromium + firefox + webkit)
|
||||
needs: pick
|
||||
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Harden the runner (Audit all outbound calls)
|
||||
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
|
||||
@@ -57,6 +53,49 @@ jobs:
|
||||
path: frontend/playwright-report/
|
||||
retention-days: 14
|
||||
|
||||
# Whole-suite accessibility sweep. Pull requests only scan the stories they
|
||||
# touch (frontend-a11y.yml) because a full pass takes ~30 minutes; this covers
|
||||
# everything else, so a violation introduced by a change somewhere other than
|
||||
# the story itself — a shared component, a theme token — still surfaces within
|
||||
# a day.
|
||||
a11y-all-stories:
|
||||
name: a11y (every story)
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 60
|
||||
steps:
|
||||
- name: Harden the runner (Audit all outbound calls)
|
||||
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
|
||||
with:
|
||||
egress-policy: audit
|
||||
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
|
||||
- name: Set up Node.js
|
||||
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
|
||||
with:
|
||||
node-version: "22"
|
||||
cache: "npm"
|
||||
cache-dependency-path: frontend/package-lock.json
|
||||
|
||||
- name: Install Task
|
||||
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
|
||||
|
||||
- name: a11y gate (every story)
|
||||
run: task frontend:storybook:a11y
|
||||
|
||||
- name: Upload scan reports
|
||||
if: always()
|
||||
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
|
||||
with:
|
||||
name: a11y-scan-nightly-${{ github.run_id }}
|
||||
path: frontend/.a11y-scan/
|
||||
retention-days: 14
|
||||
if-no-files-found: ignore
|
||||
# The reports live in a dot-directory, which upload-artifact treats as
|
||||
# hidden and silently skips by default.
|
||||
include-hidden-files: true
|
||||
|
||||
# Builds all desktop platforms on a schedule so the Rust dependency cache is
|
||||
# written on main, where PR and merge-queue tauri builds can restore it.
|
||||
warm-tauri-cache:
|
||||
|
||||
@@ -0,0 +1,159 @@
|
||||
name: PR conflict labeler
|
||||
|
||||
on:
|
||||
pull_request_target:
|
||||
types:
|
||||
- opened
|
||||
- reopened
|
||||
- synchronize
|
||||
- edited
|
||||
- ready_for_review
|
||||
schedule:
|
||||
- cron: "17 */6 * * *"
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: pr-conflict-labeler-${{ github.event.pull_request.number || 'all-open-prs' }}
|
||||
cancel-in-progress: false
|
||||
|
||||
env:
|
||||
CONFLICT_LABEL: "has conflicts"
|
||||
|
||||
jobs:
|
||||
label-conflicts:
|
||||
name: Label conflicted PRs
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: read
|
||||
issues: write
|
||||
pull-requests: read
|
||||
steps:
|
||||
- name: Harden Runner
|
||||
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
|
||||
with:
|
||||
egress-policy: audit
|
||||
|
||||
- name: Check out the repository
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
|
||||
- name: Set up stirling-bot token
|
||||
id: setup-bot
|
||||
uses: ./.github/actions/setup-bot
|
||||
with:
|
||||
app-id: ${{ secrets.GH_APP_ID }}
|
||||
private-key: ${{ secrets.GH_APP_PRIVATE_KEY }}
|
||||
|
||||
- name: Apply conflict label
|
||||
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
|
||||
with:
|
||||
github-token: ${{ steps.setup-bot.outputs.token }}
|
||||
script: |
|
||||
const conflictLabel = process.env.CONFLICT_LABEL;
|
||||
const owner = context.repo.owner;
|
||||
const repo = context.repo.repo;
|
||||
const eventPullRequest = context.payload.pull_request;
|
||||
|
||||
async function sleep(ms) {
|
||||
await new Promise((resolve) => setTimeout(resolve, ms));
|
||||
}
|
||||
|
||||
async function getPullRequestWithMergeableState(pullNumber) {
|
||||
for (let attempt = 1; attempt <= 6; attempt += 1) {
|
||||
const { data: pull } = await github.rest.pulls.get({
|
||||
owner,
|
||||
repo,
|
||||
pull_number: pullNumber,
|
||||
});
|
||||
|
||||
if (pull.mergeable !== null) {
|
||||
return pull;
|
||||
}
|
||||
|
||||
core.info(`PR #${pullNumber}: mergeable is not ready yet (attempt ${attempt}/6).`);
|
||||
await sleep(5000);
|
||||
}
|
||||
|
||||
const { data: pull } = await github.rest.pulls.get({
|
||||
owner,
|
||||
repo,
|
||||
pull_number: pullNumber,
|
||||
});
|
||||
return pull;
|
||||
}
|
||||
|
||||
async function ensureConflictLabel() {
|
||||
try {
|
||||
await github.rest.issues.getLabel({
|
||||
owner,
|
||||
repo,
|
||||
name: conflictLabel,
|
||||
});
|
||||
} catch (error) {
|
||||
if (error.status !== 404) {
|
||||
throw error;
|
||||
}
|
||||
|
||||
await github.rest.issues.createLabel({
|
||||
owner,
|
||||
repo,
|
||||
name: conflictLabel,
|
||||
color: 'D93F0B',
|
||||
description: 'Pull request has merge conflicts with the base branch',
|
||||
});
|
||||
core.info(`Created '${conflictLabel}' label.`);
|
||||
}
|
||||
}
|
||||
|
||||
async function labelPullRequest(pull) {
|
||||
const existingLabels = pull.labels.map((label) => label.name);
|
||||
const hasConflictLabel = existingLabels.includes(conflictLabel);
|
||||
const hasConflicts = pull.mergeable === false && pull.mergeable_state === 'dirty';
|
||||
|
||||
if (hasConflicts && !hasConflictLabel) {
|
||||
await github.rest.issues.addLabels({
|
||||
owner,
|
||||
repo,
|
||||
issue_number: pull.number,
|
||||
labels: [conflictLabel],
|
||||
});
|
||||
core.info(`Added '${conflictLabel}' to PR #${pull.number}.`);
|
||||
return;
|
||||
}
|
||||
|
||||
if (!hasConflicts && hasConflictLabel) {
|
||||
await github.rest.issues.removeLabel({
|
||||
owner,
|
||||
repo,
|
||||
issue_number: pull.number,
|
||||
name: conflictLabel,
|
||||
});
|
||||
core.info(`Removed '${conflictLabel}' from PR #${pull.number}.`);
|
||||
return;
|
||||
}
|
||||
|
||||
core.info(`PR #${pull.number}: no label change needed (mergeable=${pull.mergeable}, mergeable_state=${pull.mergeable_state}).`);
|
||||
}
|
||||
|
||||
await ensureConflictLabel();
|
||||
|
||||
let pullNumbers;
|
||||
if (eventPullRequest) {
|
||||
pullNumbers = [eventPullRequest.number];
|
||||
} else {
|
||||
const pulls = await github.paginate(github.rest.pulls.list, {
|
||||
owner,
|
||||
repo,
|
||||
state: 'open',
|
||||
per_page: 100,
|
||||
});
|
||||
pullNumbers = pulls.map((pull) => pull.number);
|
||||
core.info(`Checking ${pullNumbers.length} open PR(s).`);
|
||||
}
|
||||
|
||||
for (const pullNumber of pullNumbers) {
|
||||
const pull = await getPullRequestWithMergeableState(pullNumber);
|
||||
await labelPullRequest(pull);
|
||||
}
|
||||
@@ -28,6 +28,7 @@ jobs:
|
||||
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
|
||||
with:
|
||||
enable-cache: true
|
||||
cache-suffix: pre-commit
|
||||
|
||||
- name: Install Task
|
||||
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
|
||||
|
||||
@@ -13,6 +13,11 @@ on:
|
||||
required: false
|
||||
type: boolean
|
||||
default: true
|
||||
build_engine:
|
||||
description: "Build & push the stirling-pdf-engine image (plus the -docparse addon variant)."
|
||||
required: false
|
||||
type: boolean
|
||||
default: false
|
||||
force_unoserver_rebuild:
|
||||
description: "Rebuild stirling-unoserver even if its source hash is unchanged."
|
||||
required: false
|
||||
@@ -51,6 +56,8 @@ jobs:
|
||||
env:
|
||||
RUN_MAIN_APP: ${{ github.event_name != 'workflow_dispatch' || inputs.build_main_app }}
|
||||
RUN_UNOSERVER: ${{ github.event_name != 'workflow_dispatch' || inputs.build_unoserver }}
|
||||
# Engine images are dispatch-only for now; flip the default once the addon stabilises.
|
||||
RUN_ENGINE: ${{ github.event_name == 'workflow_dispatch' && inputs.build_engine }}
|
||||
steps:
|
||||
- name: Harden Runner
|
||||
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
|
||||
@@ -76,9 +83,9 @@ jobs:
|
||||
gradle-${{ runner.os }}-
|
||||
|
||||
- name: Setup Gradle
|
||||
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
|
||||
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
|
||||
with:
|
||||
gradle-version: 9.6.0
|
||||
gradle-version: 9.6.1
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
id: buildx
|
||||
@@ -139,7 +146,6 @@ jobs:
|
||||
tags: |
|
||||
type=raw,value=${{ steps.versionNumber.outputs.versionNumber }},enable=${{ github.ref == 'refs/heads/master' || github.ref == 'refs/heads/V2-master' }}
|
||||
type=raw,value=latest,enable=${{ github.ref == 'refs/heads/master' || github.ref == 'refs/heads/V2-master' }}
|
||||
type=raw,value=alpha,enable=${{ github.ref == 'refs/heads/main' || github.ref == 'refs/heads/testMain' }}
|
||||
|
||||
- name: Build and push Unified Dockerfile (latest variant)
|
||||
id: build-push-latest
|
||||
@@ -220,6 +226,62 @@ jobs:
|
||||
cosign sign --key env://COSIGN_PRIVATE_KEY --yes "${tag}@${DIGEST}"
|
||||
done
|
||||
|
||||
- name: Generate tags for engine
|
||||
id: meta-engine
|
||||
uses: docker/metadata-action@80c7e94dd9b9319bd5eb7a0e0fe9291e23a2a2e9 # v6.1.0
|
||||
if: env.RUN_ENGINE == 'true'
|
||||
with:
|
||||
images: |
|
||||
ghcr.io/${{ steps.repoowner.outputs.lowercase }}/stirling-pdf-engine
|
||||
${{ secrets.DOCKER_HUB_ORG_USERNAME }}/stirling-pdf-engine
|
||||
tags: |
|
||||
type=raw,value=${{ steps.versionNumber.outputs.versionNumber }}
|
||||
type=raw,value=latest
|
||||
|
||||
- name: Build and push engine image
|
||||
id: build-push-engine
|
||||
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
|
||||
if: env.RUN_ENGINE == 'true' && steps.meta-engine.outputs.tags != ''
|
||||
with:
|
||||
builder: ${{ steps.buildx.outputs.name }}
|
||||
context: ./engine
|
||||
push: true
|
||||
cache-from: type=gha,scope=stirling-pdf-engine
|
||||
cache-to: type=gha,mode=max,scope=stirling-pdf-engine
|
||||
tags: ${{ steps.meta-engine.outputs.tags }}
|
||||
labels: ${{ steps.meta-engine.outputs.labels }}
|
||||
platforms: linux/amd64,linux/arm64/v8
|
||||
provenance: true
|
||||
sbom: true
|
||||
|
||||
- name: Generate tags for engine docparse addon
|
||||
id: meta-engine-docparse
|
||||
uses: docker/metadata-action@80c7e94dd9b9319bd5eb7a0e0fe9291e23a2a2e9 # v6.1.0
|
||||
if: env.RUN_ENGINE == 'true'
|
||||
with:
|
||||
images: |
|
||||
ghcr.io/${{ steps.repoowner.outputs.lowercase }}/stirling-pdf-engine
|
||||
${{ secrets.DOCKER_HUB_ORG_USERNAME }}/stirling-pdf-engine
|
||||
tags: |
|
||||
type=raw,value=${{ steps.versionNumber.outputs.versionNumber }}-docparse
|
||||
type=raw,value=latest-docparse
|
||||
|
||||
- name: Build and push engine docparse addon image
|
||||
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
|
||||
if: env.RUN_ENGINE == 'true' && steps.meta-engine-docparse.outputs.tags != ''
|
||||
with:
|
||||
builder: ${{ steps.buildx.outputs.name }}
|
||||
context: ./engine
|
||||
push: true
|
||||
cache-from: type=gha,scope=stirling-pdf-engine-docparse
|
||||
cache-to: type=gha,mode=max,scope=stirling-pdf-engine-docparse
|
||||
tags: ${{ steps.meta-engine-docparse.outputs.tags }}
|
||||
labels: ${{ steps.meta-engine-docparse.outputs.labels }}
|
||||
build-args: DOCPARSE=true
|
||||
platforms: linux/amd64
|
||||
provenance: true
|
||||
sbom: true
|
||||
|
||||
- name: Generate tags for ultra-lite
|
||||
id: meta-lite
|
||||
uses: docker/metadata-action@80c7e94dd9b9319bd5eb7a0e0fe9291e23a2a2e9 # v6.1.0
|
||||
|
||||
@@ -22,15 +22,9 @@ permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
pick:
|
||||
uses: ./.github/workflows/_runner-pick.yml
|
||||
|
||||
push:
|
||||
if: ${{ vars.CI_PROFILE != 'lite' }}
|
||||
needs: pick
|
||||
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
|
||||
env:
|
||||
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Harden Runner
|
||||
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
|
||||
@@ -46,9 +40,9 @@ jobs:
|
||||
distribution: "temurin"
|
||||
|
||||
- name: Setup Gradle
|
||||
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
|
||||
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
|
||||
with:
|
||||
gradle-version: 9.6.0
|
||||
gradle-version: 9.6.1
|
||||
|
||||
- name: Generate Swagger documentation
|
||||
run: ./gradlew :stirling-pdf:generateOpenApiDocs
|
||||
|
||||
@@ -0,0 +1,95 @@
|
||||
name: Sync Portal Docs
|
||||
|
||||
# Regenerates the portal Developer Docs manifest from the Stirling docs repo and
|
||||
# opens a PR when it changes. Runs weekly, on manual dispatch, or when the docs
|
||||
# repo fires a `docs-updated` repository_dispatch.
|
||||
on:
|
||||
schedule:
|
||||
- cron: "0 6 * * 1"
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
ref:
|
||||
description: "Docs repo ref (branch or tag) to sync from"
|
||||
required: false
|
||||
default: "main"
|
||||
repository_dispatch:
|
||||
types: [docs-updated]
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}
|
||||
cancel-in-progress: true
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
sync:
|
||||
name: Sync docs manifest
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
steps:
|
||||
- name: Harden Runner
|
||||
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
|
||||
with:
|
||||
egress-policy: audit
|
||||
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
with:
|
||||
fetch-depth: 0
|
||||
persist-credentials: false
|
||||
|
||||
- name: Setup GitHub App Bot
|
||||
id: setup-bot
|
||||
uses: ./.github/actions/setup-bot
|
||||
with:
|
||||
app-id: ${{ secrets.GH_APP_ID }}
|
||||
private-key: ${{ secrets.GH_APP_PRIVATE_KEY }}
|
||||
|
||||
- name: Set up Node.js
|
||||
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
|
||||
with:
|
||||
node-version: "22"
|
||||
cache: "npm"
|
||||
cache-dependency-path: frontend/package-lock.json
|
||||
|
||||
- name: Install frontend dependencies
|
||||
working-directory: frontend
|
||||
env:
|
||||
NPM_CONFIG_IGNORE_SCRIPTS: "true"
|
||||
run: npm ci --ignore-scripts --audit=false --fund=false
|
||||
|
||||
- name: Regenerate docs manifest
|
||||
working-directory: frontend
|
||||
env:
|
||||
DOCS_REF: ${{ github.event.inputs.ref || github.event.client_payload.ref || 'main' }}
|
||||
GITHUB_TOKEN: ${{ steps.setup-bot.outputs.token }}
|
||||
run: npm run docs:sync
|
||||
|
||||
- name: Create Pull Request
|
||||
id: cpr
|
||||
uses: peter-evans/create-pull-request@5f6978faf089d4d20b00c7766989d076bb2fc7f1 # v8.1.1
|
||||
with:
|
||||
token: ${{ steps.setup-bot.outputs.token }}
|
||||
commit-message: "Sync portal docs from docs repo"
|
||||
committer: ${{ steps.setup-bot.outputs.committer }}
|
||||
author: ${{ steps.setup-bot.outputs.committer }}
|
||||
signoff: true
|
||||
branch: sync-portal-docs
|
||||
base: main
|
||||
title: "Sync portal docs from docs repo"
|
||||
body: |
|
||||
Auto-generated by ${{ steps.setup-bot.outputs.app-slug }}[bot].
|
||||
|
||||
Regenerates `frontend/editor/src/portal/generated/docsManifest.json`
|
||||
from the Stirling docs repo via `npm run docs:sync`.
|
||||
labels: |
|
||||
Documentation
|
||||
github-actions
|
||||
Front End
|
||||
add-paths: frontend/editor/src/portal/generated/docsManifest.json
|
||||
delete-branch: true
|
||||
sign-commits: true
|
||||
@@ -10,6 +10,7 @@ on:
|
||||
- "app/common/build.gradle"
|
||||
- "app/core/build.gradle"
|
||||
- "app/proprietary/build.gradle"
|
||||
- "gradle/spotless.gradle"
|
||||
- "README.md"
|
||||
- "frontend/editor/public/locales/*/translation.toml"
|
||||
- "app/core/src/main/resources/static/3rdPartyLicenses.json"
|
||||
@@ -51,7 +52,7 @@ jobs:
|
||||
private-key: ${{ secrets.GH_APP_PRIVATE_KEY }}
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
|
||||
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
|
||||
with:
|
||||
python-version: "3.12"
|
||||
cache: "pip" # caching pip dependencies
|
||||
@@ -64,6 +65,7 @@ jobs:
|
||||
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
|
||||
with:
|
||||
enable-cache: true
|
||||
cache-suffix: sync-files
|
||||
|
||||
- name: Install Task
|
||||
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
|
||||
|
||||
@@ -12,7 +12,7 @@ on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
platform:
|
||||
description: "Platform to build (windows, macos, linux, or all)."
|
||||
description: "Platform to build (windows, macos, linux, windows-macos, or all)."
|
||||
required: false
|
||||
type: string
|
||||
default: "all"
|
||||
@@ -29,7 +29,7 @@ on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
platform:
|
||||
description: "Platform to build (windows, macos, linux, or all)"
|
||||
description: "Platform to build (windows, macos, linux, windows-macos, or all)"
|
||||
required: true
|
||||
default: "all"
|
||||
type: choice
|
||||
@@ -38,6 +38,7 @@ on:
|
||||
- windows
|
||||
- macos
|
||||
- linux
|
||||
- windows-macos
|
||||
sign:
|
||||
description: "Sign and notarize the bundles."
|
||||
required: false
|
||||
@@ -76,10 +77,11 @@ jobs:
|
||||
LINUX='{"platform":"ubuntu-22.04","args":"","name":"linux-x86_64","jpdfium_platforms":"linux-x64"}'
|
||||
|
||||
case "$PLATFORM" in
|
||||
windows) ENTRIES=("$WINDOWS") ;;
|
||||
macos) ENTRIES=("$MACOS") ;;
|
||||
linux) ENTRIES=("$LINUX") ;;
|
||||
*) ENTRIES=("$WINDOWS" "$MACOS" "$LINUX") ;;
|
||||
windows) ENTRIES=("$WINDOWS") ;;
|
||||
macos) ENTRIES=("$MACOS") ;;
|
||||
linux) ENTRIES=("$LINUX") ;;
|
||||
windows-macos) ENTRIES=("$WINDOWS" "$MACOS") ;;
|
||||
*) ENTRIES=("$WINDOWS" "$MACOS" "$LINUX") ;;
|
||||
esac
|
||||
|
||||
# Drop macOS entries when Apple certificate secret is unavailable
|
||||
@@ -106,7 +108,6 @@ jobs:
|
||||
WINDOWS_CERTIFICATE: ${{ secrets.WINDOWS_CERTIFICATE }}
|
||||
APPLE_CERTIFICATE: ${{ secrets.APPLE_CERTIFICATE }}
|
||||
RELEASE_GPG_PRIVATE_KEY: ${{ secrets.RELEASE_GPG_PRIVATE_KEY }}
|
||||
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
|
||||
steps:
|
||||
- name: Harden Runner
|
||||
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
|
||||
@@ -168,9 +169,9 @@ jobs:
|
||||
distribution: "temurin"
|
||||
|
||||
- name: Setup Gradle
|
||||
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
|
||||
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
|
||||
with:
|
||||
gradle-version: 9.6.0
|
||||
gradle-version: 9.6.1
|
||||
|
||||
- name: Setup Task
|
||||
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
|
||||
|
||||
@@ -17,19 +17,11 @@ on:
|
||||
required: false
|
||||
type: string
|
||||
default: "false"
|
||||
depot_cores:
|
||||
description: "Depot runner vCPU count (used in runs-on). Override for benchmarking."
|
||||
required: false
|
||||
type: string
|
||||
default: "8"
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
pick:
|
||||
uses: ./.github/workflows/_runner-pick.yml
|
||||
|
||||
# TODO: extract a pre-matrix `prepare` job that runs once and produces
|
||||
# shared artifacts for the three matrix entries below to consume:
|
||||
# 1. `task backend:build` — currently runs 3× in parallel with
|
||||
@@ -45,14 +37,7 @@ jobs:
|
||||
# spring-security=true matrix entry if `task backend:build` and
|
||||
# `task backend:build:ci` produce equivalent JARs (verify before wiring).
|
||||
test-build-docker-images:
|
||||
needs: pick
|
||||
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && format('depot-ubuntu-24.04-{0}', inputs.depot_cores || '8') || 'ubuntu-latest' }}
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
env:
|
||||
USE_DEPOT: ${{ needs.pick.outputs.use_depot == 'true' && inputs.docker-base-changed != 'true' }}
|
||||
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
@@ -109,9 +94,9 @@ jobs:
|
||||
key: gradle-deps-${{ runner.os }}-jdk-25-${{ hashFiles('**/gradle/wrapper/gradle-wrapper.properties', '**/*.gradle', '**/*.gradle.kts', 'settings.gradle', 'settings.gradle.kts', 'gradle/libs.versions.toml') }}
|
||||
|
||||
- name: Setup Gradle
|
||||
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
|
||||
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
|
||||
with:
|
||||
gradle-version: 9.6.0
|
||||
gradle-version: 9.6.1
|
||||
cache-disabled: true
|
||||
|
||||
- name: Install Task
|
||||
@@ -125,16 +110,10 @@ jobs:
|
||||
DISABLE_ADDITIONAL_FEATURES: true
|
||||
STIRLING_PDF_DESKTOP_UI: false
|
||||
|
||||
- name: Set up Depot CLI
|
||||
if: env.USE_DEPOT == 'true'
|
||||
uses: depot/setup-action@15c09a5f77a0840ad4bce955686522a257853461 # v1.0.0
|
||||
|
||||
- name: Set up QEMU
|
||||
if: env.USE_DEPOT != 'true'
|
||||
uses: docker/setup-qemu-action@ce360397dd3f832beb865e1373c09c0e9f86d70a # v4.0.0
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
if: env.USE_DEPOT != 'true'
|
||||
id: buildx
|
||||
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
|
||||
|
||||
@@ -182,24 +161,10 @@ jobs:
|
||||
--tag stirling-pdf-embedded:pr-test \
|
||||
.
|
||||
|
||||
- name: Build ${{ matrix.docker-rev }} (Depot)
|
||||
if: env.USE_DEPOT == 'true'
|
||||
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.16.0
|
||||
with:
|
||||
project: ${{ vars.DEPOT_PROJECT_ID }}
|
||||
context: .
|
||||
file: ./${{ matrix.docker-rev }}
|
||||
push: false
|
||||
platforms: ${{ steps.build-params.outputs.platforms }}
|
||||
build-args: |
|
||||
BASE_IMAGE=${{ steps.build-params.outputs.base_image }}
|
||||
provenance: true
|
||||
sbom: true
|
||||
|
||||
# Fork PRs that did NOT change the base use the buildx container builder
|
||||
# PRs that did NOT change the base use the buildx container builder
|
||||
# (multi-platform + gha cache) against the published base image.
|
||||
- name: Build ${{ matrix.docker-rev }} (Docker fork fallback)
|
||||
if: env.USE_DEPOT != 'true' && inputs.docker-base-changed != 'true'
|
||||
- name: Build ${{ matrix.docker-rev }}
|
||||
if: inputs.docker-base-changed != 'true'
|
||||
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
|
||||
with:
|
||||
builder: ${{ steps.buildx.outputs.name }}
|
||||
@@ -227,14 +192,7 @@ jobs:
|
||||
if-no-files-found: warn
|
||||
|
||||
test-build-unoserver-image:
|
||||
needs: pick
|
||||
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && format('depot-ubuntu-24.04-{0}', inputs.depot_cores || '8') || 'ubuntu-latest' }}
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
env:
|
||||
USE_DEPOT: ${{ needs.pick.outputs.use_depot == 'true' && inputs.docker-base-changed != 'true' }}
|
||||
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Harden Runner
|
||||
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
|
||||
@@ -244,35 +202,14 @@ jobs:
|
||||
- name: Checkout Repository
|
||||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||||
|
||||
- name: Set up Depot CLI
|
||||
if: env.USE_DEPOT == 'true'
|
||||
uses: depot/setup-action@15c09a5f77a0840ad4bce955686522a257853461 # v1.0.0
|
||||
|
||||
- name: Set up QEMU
|
||||
if: env.USE_DEPOT != 'true'
|
||||
uses: docker/setup-qemu-action@ce360397dd3f832beb865e1373c09c0e9f86d70a # v4.0.0
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
if: env.USE_DEPOT != 'true'
|
||||
id: buildx
|
||||
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
|
||||
|
||||
- name: Build docker/unoserver/Dockerfile (Depot)
|
||||
if: env.USE_DEPOT == 'true'
|
||||
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.16.0
|
||||
with:
|
||||
project: ${{ vars.DEPOT_PROJECT_ID }}
|
||||
context: .
|
||||
file: ./docker/unoserver/Dockerfile
|
||||
push: false
|
||||
load: true
|
||||
platforms: linux/amd64
|
||||
tags: stirling-unoserver:pr-test
|
||||
provenance: false
|
||||
sbom: false
|
||||
|
||||
- name: Build docker/unoserver/Dockerfile (Docker fork fallback)
|
||||
if: env.USE_DEPOT != 'true'
|
||||
- name: Build docker/unoserver/Dockerfile
|
||||
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
|
||||
with:
|
||||
builder: ${{ steps.buildx.outputs.name }}
|
||||
|
||||
@@ -20,19 +20,9 @@ permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
pick:
|
||||
uses: ./.github/workflows/_runner-pick.yml
|
||||
|
||||
deploy:
|
||||
if: ${{ vars.CI_PROFILE != 'lite' }}
|
||||
needs: pick
|
||||
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
env:
|
||||
USE_DEPOT: ${{ needs.pick.outputs.use_depot == 'true' }}
|
||||
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Harden Runner
|
||||
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
|
||||
@@ -49,9 +39,9 @@ jobs:
|
||||
distribution: "temurin"
|
||||
|
||||
- name: Setup Gradle
|
||||
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
|
||||
uses: gradle/actions/setup-gradle@3f131e8634966bd73d06cc69884922b02e6faf92 # v6.2.0
|
||||
with:
|
||||
gradle-version: 9.6.0
|
||||
gradle-version: 9.6.1
|
||||
|
||||
- name: Build with Gradle
|
||||
run: ./gradlew build
|
||||
@@ -61,12 +51,7 @@ jobs:
|
||||
MAVEN_PUBLIC_URL: ${{ secrets.MAVEN_PUBLIC_URL }}
|
||||
DISABLE_ADDITIONAL_FEATURES: true
|
||||
|
||||
- name: Set up Depot CLI
|
||||
if: env.USE_DEPOT == 'true'
|
||||
uses: depot/setup-action@15c09a5f77a0840ad4bce955686522a257853461 # v1.0.0
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
if: env.USE_DEPOT != 'true'
|
||||
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
|
||||
|
||||
- name: Get version number
|
||||
@@ -81,20 +66,7 @@ jobs:
|
||||
username: ${{ secrets.DOCKER_HUB_USERNAME }}
|
||||
password: ${{ secrets.DOCKER_HUB_API }}
|
||||
|
||||
- name: Build and push test image (Depot)
|
||||
if: env.USE_DEPOT == 'true'
|
||||
uses: depot/build-push-action@98e78adca7817480b8185f474a400b451d74e287 # v1.16.0
|
||||
with:
|
||||
project: ${{ vars.DEPOT_PROJECT_ID }}
|
||||
context: .
|
||||
file: ./docker/embedded/Dockerfile
|
||||
push: true
|
||||
tags: ${{ secrets.DOCKER_HUB_USERNAME }}/test:test-${{ github.sha }}
|
||||
build-args: VERSION_TAG=${{ steps.versionNumber.outputs.versionNumber }}
|
||||
platforms: linux/amd64
|
||||
|
||||
- name: Build and push test image (Docker fork fallback)
|
||||
if: env.USE_DEPOT != 'true'
|
||||
- name: Build and push test image
|
||||
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
|
||||
with:
|
||||
context: .
|
||||
@@ -153,8 +125,7 @@ jobs:
|
||||
files-changed:
|
||||
if: always()
|
||||
name: detect what files changed
|
||||
needs: pick
|
||||
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 3
|
||||
outputs:
|
||||
frontend: ${{ steps.changes.outputs.frontend }}
|
||||
@@ -174,8 +145,8 @@ jobs:
|
||||
|
||||
test:
|
||||
if: needs.files-changed.outputs.frontend == 'true'
|
||||
needs: [pick, deploy, files-changed]
|
||||
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
|
||||
needs: [deploy, files-changed]
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Harden Runner
|
||||
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
|
||||
@@ -208,8 +179,8 @@ jobs:
|
||||
FORCE_COLOR: "3"
|
||||
|
||||
cleanup:
|
||||
needs: [pick, deploy, test]
|
||||
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
|
||||
needs: [deploy, test]
|
||||
runs-on: ubuntu-latest
|
||||
if: always()
|
||||
|
||||
steps:
|
||||
|
||||
+67
-4
@@ -85,7 +85,8 @@ tasks:
|
||||
# full path, hostname, or user. Consumed at dev-serve time by vite.config
|
||||
# and dropped from production builds.
|
||||
STIRLING_DEV_LABEL:
|
||||
sh: basename "$(git rev-parse --show-toplevel 2>/dev/null || pwd)"
|
||||
sh: >-
|
||||
{{if eq OS "windows"}}powershell -NoProfile -Command '$root = git rev-parse --show-toplevel 2>$null; if (-not $root) { $root = (Get-Location).Path }; Split-Path -Leaf $root'{{else}}basename "$(git rev-parse --show-toplevel 2>/dev/null || pwd)"{{end}}
|
||||
cmds:
|
||||
- npx vite editor --mode {{.MODE}} --port {{.PORT}}{{if .OPEN}} --open{{end}}
|
||||
|
||||
@@ -183,16 +184,74 @@ tasks:
|
||||
|
||||
storybook:
|
||||
desc: "Start Storybook dev server"
|
||||
deps: [install]
|
||||
deps: [prepare]
|
||||
cmds:
|
||||
- npx storybook dev -p 6006 {{.CLI_ARGS}}
|
||||
|
||||
storybook:build:
|
||||
desc: "Build static Storybook"
|
||||
deps: [install]
|
||||
deps: [prepare]
|
||||
cmds:
|
||||
- npx storybook build {{.CLI_ARGS}}
|
||||
|
||||
storybook:browser:
|
||||
internal: true
|
||||
desc: "Install the Chromium build the story scan runs in"
|
||||
run: once
|
||||
deps: [install]
|
||||
cmds:
|
||||
- npx playwright install chromium
|
||||
|
||||
storybook:test:
|
||||
desc: "Scan every story in real Chromium: it must render and pass axe"
|
||||
deps: [prepare, storybook:browser]
|
||||
cmds:
|
||||
# Runs each story as a browser test. Pass a filter through, e.g.
|
||||
# task frontend:storybook:test -- Button
|
||||
- npx vitest run --config .storybook/vitest.config.ts {{.CLI_ARGS}}
|
||||
|
||||
storybook:a11y:
|
||||
desc: "a11y regression gate over every story: fail only on NEW axe violations"
|
||||
deps: [prepare, storybook:browser]
|
||||
cmds:
|
||||
- node .storybook/a11y-scan.mjs
|
||||
- node .storybook/a11y-check.mjs --in .a11y-scan --manifest .a11y-scan/manifest.txt
|
||||
|
||||
storybook:a11y:changed:
|
||||
desc: "a11y gate over the stories this branch affects (default base origin/main)"
|
||||
summary: |
|
||||
Scans the stories a branch affects, which is what pull requests run — a
|
||||
full scan takes ~30 minutes, far too long to sit in front of every merge.
|
||||
A story is affected if its file changed, or if a same-named sibling
|
||||
source file changed (editing Button.tsx or Button.css re-scans
|
||||
Button.stories.tsx — the story renders the live component, so a component
|
||||
edit changes what the story shows without touching the story file).
|
||||
Changes that ripple further than a component's own stories are covered by
|
||||
the nightly full sweep.
|
||||
|
||||
Pass a base ref through CLI_ARGS, e.g.
|
||||
task frontend:storybook:a11y:changed -- origin/release
|
||||
deps: [prepare, storybook:browser]
|
||||
vars:
|
||||
BASE: '{{.CLI_ARGS | default "origin/main"}}'
|
||||
CHANGED:
|
||||
sh: node .storybook/a11y-changed.mjs {{.CLI_ARGS | default "origin/main"}}
|
||||
cmds:
|
||||
- cmd: |
|
||||
if [ -z '{{.CHANGED}}' ]; then
|
||||
echo "a11y: no story files affected vs {{.BASE}} — nothing to check"
|
||||
exit 0
|
||||
fi
|
||||
node .storybook/a11y-scan.mjs {{.CHANGED}}
|
||||
node .storybook/a11y-check.mjs --in .a11y-scan --manifest .a11y-scan/manifest.txt
|
||||
|
||||
storybook:a11y:record:
|
||||
desc: "Re-record the a11y baseline (run after intentionally fixing/adding violations)"
|
||||
deps: [prepare, storybook:browser]
|
||||
cmds:
|
||||
- node .storybook/a11y-scan.mjs
|
||||
- node .storybook/a11y-check.mjs --in .a11y-scan --manifest .a11y-scan/manifest.txt --record
|
||||
|
||||
# ============================================================
|
||||
# Code quality
|
||||
# ============================================================
|
||||
@@ -206,10 +265,14 @@ tasks:
|
||||
- task: lint:colors
|
||||
|
||||
lint:colors:
|
||||
desc: "Enforce theme tokens — colours in core/theme route through the palette"
|
||||
desc: "Enforce theme tokens — no hardcoded colours or raw primitives in components"
|
||||
aliases: [lint:colours]
|
||||
deps: [install]
|
||||
cmds:
|
||||
- node editor/scripts/lint/theme-lint.mjs
|
||||
- node editor/scripts/lint/theme-lint.mjs css-colors
|
||||
- node editor/scripts/lint/theme-lint.mjs code-colors
|
||||
- node editor/scripts/lint/theme-lint.mjs no-primitives
|
||||
|
||||
contrast:
|
||||
desc: "Report low-contrast theme token pairs (warning only, never blocks)"
|
||||
|
||||
@@ -73,7 +73,7 @@ tasks:
|
||||
- task: gitleaks
|
||||
|
||||
install:
|
||||
desc: "Install the pinned pre-commit Python tools (ruff, codespell, toml-sort)"
|
||||
desc: "Install the pinned pre-commit Python tools"
|
||||
run: once
|
||||
cmds:
|
||||
- uv sync --project scripts/pre-commit --locked
|
||||
@@ -112,7 +112,7 @@ tasks:
|
||||
toml-sort:
|
||||
deps: [install]
|
||||
cmds:
|
||||
- uv run --project scripts/pre-commit --no-sync toml-sort --all --ignore-case {{if .FIX}}--in-place{{else}}--check{{end}} {{.LOCALE_TOML}}
|
||||
- uv run --project scripts/pre-commit --no-sync python scripts/pre-commit/sort_locale_toml.py {{if .FIX}}--fix {{end}}{{.LOCALE_TOML}}
|
||||
|
||||
whitespace:
|
||||
cmds:
|
||||
|
||||
@@ -22,6 +22,8 @@ if that directory exists, is licensed under the license defined in "frontend/edi
|
||||
if that directory exists, is licensed under the license defined in "frontend/editor/src/prototypes/LICENSE".
|
||||
* All content that resides under the "frontend/editor/src/portal/" directory of this repository,
|
||||
if that directory exists, is licensed under the license defined in "frontend/editor/src/portal/LICENSE".
|
||||
* All content that resides under the "frontend/editor/src/portal-saas/" directory of this repository,
|
||||
if that directory exists, is licensed under the license defined in "frontend/editor/src/portal-saas/LICENSE".
|
||||
* Content outside of the above mentioned directories or restrictions above is
|
||||
available under the MIT License as defined below.
|
||||
|
||||
|
||||
+1
-27
@@ -2,32 +2,6 @@
|
||||
bootRun {
|
||||
enabled = false
|
||||
}
|
||||
spotless {
|
||||
java {
|
||||
target 'src/**/java/**/*.java'
|
||||
targetExclude 'src/main/java/org/apache/**'
|
||||
googleJavaFormat(googleJavaFormatVersion).aosp().reorderImports(false)
|
||||
// google-java-format 1.28.0 bundles Guava 32.x which crashes Spotless lint on JDK 24/25
|
||||
suppressLintsFor { setStep('google-java-format') }
|
||||
|
||||
importOrder("java", "javax", "org", "com", "net", "io", "jakarta", "lombok", "me", "stirling")
|
||||
trimTrailingWhitespace()
|
||||
leadingTabsToSpaces()
|
||||
endWithNewline()
|
||||
}
|
||||
yaml {
|
||||
target '**/*.yml', '**/*.yaml'
|
||||
trimTrailingWhitespace()
|
||||
leadingTabsToSpaces()
|
||||
endWithNewline()
|
||||
}
|
||||
format 'gradle', {
|
||||
target '**/gradle/*.gradle', '**/*.gradle'
|
||||
trimTrailingWhitespace()
|
||||
leadingTabsToSpaces()
|
||||
endWithNewline()
|
||||
}
|
||||
}
|
||||
dependencies {
|
||||
api "com.google.guava:guava:${guavaVersion}"
|
||||
api 'org.springframework.boot:spring-boot-starter-webmvc'
|
||||
@@ -42,7 +16,7 @@ dependencies {
|
||||
api "org.apache.pdfbox:pdfbox-io:$pdfboxVersion"
|
||||
api "org.apache.pdfbox:xmpbox:$pdfboxVersion"
|
||||
api "org.apache.pdfbox:preflight:$pdfboxVersion"
|
||||
api 'com.github.junrar:junrar:7.5.10' // RAR archive support for CBR files
|
||||
api 'com.github.junrar:junrar:7.6.0' // RAR archive support for CBR files
|
||||
api 'jakarta.servlet:jakarta.servlet-api:6.1.0'
|
||||
api 'org.snakeyaml:snakeyaml-engine:3.0.1'
|
||||
api "org.springdoc:springdoc-openapi-starter-webmvc-ui:3.0.3"
|
||||
|
||||
@@ -433,6 +433,12 @@ public class EndpointConfiguration {
|
||||
addEndpointToGroup("Automation", "automate"); // Alias for handleData (user-friendly name)
|
||||
addEndpointToGroup("Automation", "pipeline");
|
||||
|
||||
// Adding endpoints to "DocParse" group (ingestion: chunk + index + export)
|
||||
addEndpointToGroup("DocParse", "rag-ingest");
|
||||
addEndpointToGroup("DocParse", "extract-tables");
|
||||
addEndpointToGroup("DocParse", "extract-fields");
|
||||
addEndpointToGroup("DocParse", "suggest-schema");
|
||||
|
||||
// Adding endpoints to "DeveloperTools" group
|
||||
addEndpointToGroup("DeveloperTools", "show-javascript");
|
||||
|
||||
|
||||
+142
-3
@@ -77,6 +77,7 @@ public class ApplicationProperties {
|
||||
private ProcessExecutor processExecutor = new ProcessExecutor();
|
||||
private PdfEditor pdfEditor = new PdfEditor();
|
||||
private AiEngine aiEngine = new AiEngine();
|
||||
private Docparse docparse = new Docparse();
|
||||
private Mcp mcp = new Mcp();
|
||||
private InternalApi internalApi = new InternalApi();
|
||||
private Cluster cluster = new Cluster();
|
||||
@@ -208,9 +209,10 @@ public class ApplicationProperties {
|
||||
public static class Policies {
|
||||
/**
|
||||
* Absolute directories that policy folder input sources and output sinks may read from or
|
||||
* write to. Empty (the default) disables folder access entirely, so a policy can never be
|
||||
* pointed at an arbitrary server path. Stirling's own config directory is always
|
||||
* off-limits, and folder access is always disabled in SaaS mode regardless of this list.
|
||||
* write to. Empty (the default) disables folder access except to implicitly defined
|
||||
* folders, such as server storage folders (if enabled) and the pipeline watched folders.
|
||||
* Stirling's own config directory is always off-limits, and folder access is always
|
||||
* disabled in SaaS mode regardless of this list.
|
||||
*/
|
||||
private List<String> allowedFolderRoots = new java.util.ArrayList<>();
|
||||
|
||||
@@ -249,6 +251,29 @@ public class ApplicationProperties {
|
||||
* in-network object store.
|
||||
*/
|
||||
private boolean allowPrivateS3Endpoints = false;
|
||||
|
||||
/**
|
||||
* Whether an API/Purview/ConsignO integration's base URL may resolve to a loopback,
|
||||
* link-local, or private address. Off by default: unlike S3 connections, any user may
|
||||
* create one of these, so without this gate a user could point a connection at the cloud
|
||||
* metadata address and have the server fetch it for them. Enable only when integrations
|
||||
* genuinely live inside the network (e.g. an on-prem ConsignO or an internal API gateway).
|
||||
*/
|
||||
private boolean allowPrivateApiEndpoints = false;
|
||||
|
||||
/**
|
||||
* Whether administrators may define their own API integrations - a free-form base URL,
|
||||
* path, body and headers - as opposed to only using the built-in vendor presets (Purview,
|
||||
* ConsignO, S3). On by default, and admin-only regardless: a custom integration can point
|
||||
* the server at any host, so it is authoring power, not self-serve.
|
||||
*
|
||||
* <p>Turning this off stops new custom integrations being created or edited. Ones that
|
||||
* already exist keep running, because a policy that silently stopped calling out would be a
|
||||
* worse surprise than one that keeps working; disable the connection itself to stop it.
|
||||
*/
|
||||
private boolean allowCustomApiIntegrations = true;
|
||||
|
||||
private long webhookMaxBytes = 104857600L;
|
||||
}
|
||||
|
||||
@Data
|
||||
@@ -303,6 +328,120 @@ public class ApplicationProperties {
|
||||
* explicitly requests it via {@code AiEngineClient.postWithTimeout}.
|
||||
*/
|
||||
private int longRunningTimeoutSeconds = 600;
|
||||
|
||||
/** Timeout (seconds) for the SSE stream held open by long-running orchestrator runs. */
|
||||
private int streamTimeoutSeconds = 1800;
|
||||
|
||||
/**
|
||||
* Whether the processor pushes settings-derived AI config to the engine on startup/save.
|
||||
* Pin false for env-driven deployments (SaaS) to keep the engine env-controlled.
|
||||
*/
|
||||
private boolean pushConfigToEngine = true;
|
||||
|
||||
/** Model + provider selection, forwarded to the engine per-request. */
|
||||
private Models models = new Models();
|
||||
|
||||
/** Retrieval-augmented-generation (RAG) knobs, forwarded to the engine per-request. */
|
||||
private Rag rag = new Rag();
|
||||
|
||||
/** Request size / cost guardrails. */
|
||||
private Limits limits = new Limits();
|
||||
|
||||
/** Per-capability on/off switches so an admin can disable individual AI tools. */
|
||||
private Features features = new Features();
|
||||
|
||||
@Data
|
||||
public static class Models {
|
||||
/** Provider driving the model strings: 'anthropic', 'openai', 'ollama', or 'custom'. */
|
||||
private String provider = "anthropic";
|
||||
|
||||
/** High-quality tier model name (without provider prefix), e.g. 'claude-haiku-4-5'. */
|
||||
private String smartModel = "claude-haiku-4-5";
|
||||
|
||||
/** Cheap/fast tier model name (without provider prefix). */
|
||||
private String fastModel = "claude-haiku-4-5";
|
||||
|
||||
private int smartMaxTokens = 8192;
|
||||
private int fastMaxTokens = 2048;
|
||||
|
||||
/**
|
||||
* API key for the selected provider (secret; masked). Empty means the engine uses its
|
||||
* own env credential (e.g. ANTHROPIC_API_KEY).
|
||||
*/
|
||||
private String apiKey = "";
|
||||
|
||||
/**
|
||||
* OpenAI-compatible base URL for 'ollama' / 'custom' providers (e.g.
|
||||
* http://ollama:11434/v1). Ignored for anthropic/openai. SSRF-sensitive - admin only.
|
||||
*/
|
||||
private String baseUrl = "";
|
||||
}
|
||||
|
||||
@Data
|
||||
public static class Rag {
|
||||
/**
|
||||
* Embedding provider: 'voyageai', 'openai', 'ollama', or 'custom' (OpenAI-compatible).
|
||||
*/
|
||||
private String embeddingProvider = "voyageai";
|
||||
|
||||
/** Embedding model name (without provider prefix), e.g. 'voyage-4'. */
|
||||
private String embeddingModel = "voyage-4";
|
||||
|
||||
/**
|
||||
* Secret API key for the embedding provider; masked + env-overridable like
|
||||
* models.apiKey.
|
||||
*/
|
||||
private String embeddingApiKey = "";
|
||||
|
||||
/**
|
||||
* OpenAI-compatible base URL for 'ollama' / 'custom' embedding providers (e.g.
|
||||
* http://ollama:11434/v1). Ignored for voyageai/openai. SSRF-sensitive - admin only.
|
||||
*/
|
||||
private String embeddingBaseUrl = "";
|
||||
|
||||
/** How many chunks retrieval returns per search. */
|
||||
private int topK = 20;
|
||||
|
||||
/** Per-run cap on knowledge-search tool calls before the agent must answer. */
|
||||
private int maxSearches = 5;
|
||||
}
|
||||
|
||||
@Data
|
||||
public static class Limits {
|
||||
private int maxPages = 200;
|
||||
private int maxCharacters = 200000;
|
||||
|
||||
/** Process-wide cap on concurrent model API calls (engine restart to apply). */
|
||||
private int modelMaxConcurrency = 32;
|
||||
}
|
||||
|
||||
@Data
|
||||
public static class Features {
|
||||
private boolean chat = true;
|
||||
private boolean documentQuestions = true;
|
||||
private boolean createPdf = true;
|
||||
private boolean mathAuditor = true;
|
||||
private boolean pdfComment = true;
|
||||
private boolean classify = true;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* DocParse settings (top-level {@code docparse.*}): document understanding for ingestion
|
||||
* pipelines. The basic tier (text layer) always works; the advanced tier lives in the engine's
|
||||
* docparse addon.
|
||||
*/
|
||||
@Data
|
||||
public static class Docparse {
|
||||
|
||||
/** Master switch; hides the DocParse endpoints when false. */
|
||||
private boolean enabled = true;
|
||||
|
||||
/** Requested tier: 'auto', 'basic', or 'advanced'. 'auto' resolves per document. */
|
||||
private String mode = "auto";
|
||||
|
||||
/** Mirrors DOCPARSE_AUTO_INSTALL for the engine's boot-time addon install script. */
|
||||
private boolean autoInstall = false;
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -4,6 +4,8 @@ import java.time.LocalDateTime;
|
||||
import java.util.ArrayList;
|
||||
import java.util.Collections;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
import java.util.concurrent.CopyOnWriteArrayList;
|
||||
|
||||
import com.fasterxml.jackson.annotation.JsonIgnore;
|
||||
@@ -47,6 +49,9 @@ public class JobResult {
|
||||
*/
|
||||
private final List<String> notes = new CopyOnWriteArrayList<>();
|
||||
|
||||
/** Key/value metadata that survives the write-through into the shared job store. */
|
||||
private final Map<String, String> metadata = new ConcurrentHashMap<>();
|
||||
|
||||
/**
|
||||
* Create a new JobResult with the given job ID
|
||||
*
|
||||
@@ -161,4 +166,16 @@ public class JobResult {
|
||||
public List<String> getNotes() {
|
||||
return Collections.unmodifiableList(notes);
|
||||
}
|
||||
|
||||
/** Attach a metadata value, e.g. a policy id so cluster peers can identify a policy run. */
|
||||
public void putMetadata(String key, String value) {
|
||||
if (key != null && value != null) {
|
||||
this.metadata.put(key, value);
|
||||
}
|
||||
}
|
||||
|
||||
/** An unmodifiable view of this job's metadata. */
|
||||
public Map<String, String> getMetadata() {
|
||||
return Collections.unmodifiableMap(metadata);
|
||||
}
|
||||
}
|
||||
|
||||
+16
@@ -0,0 +1,16 @@
|
||||
package stirling.software.common.service;
|
||||
|
||||
/**
|
||||
* View of the engine's DocParse capability for modules that cannot see the proprietary
|
||||
* implementation (e.g. ConfigController in core). Implemented by the proprietary
|
||||
* DocparseCapabilityService; absent when the proprietary module is not loaded.
|
||||
*/
|
||||
public interface DocparseCapabilityServiceInterface {
|
||||
|
||||
/**
|
||||
* Whether the engine reports the docparse addon (advanced tier) as installed. Must be cheap and
|
||||
* non-blocking: returns the cached probe result, {@code false} when the engine is disabled,
|
||||
* unreachable, or not yet probed.
|
||||
*/
|
||||
boolean isAdvancedInstalled();
|
||||
}
|
||||
@@ -46,9 +46,14 @@ public class InternalApiClient {
|
||||
// The second alternation carves out `/api/v1/ai/tools/*` specifically — AI tools are
|
||||
// dispatchable, but the broader `/api/v1/ai/` surface (orchestrate, health, etc.) is
|
||||
// intentionally NOT permitted to avoid plan steps re-entering the orchestrator.
|
||||
//
|
||||
// `/api/v1/integration/*` holds third-party steps (external API call, Purview labelling,
|
||||
// ConsignO). They reach outside the JVM, so the namespace is deliberately kept to tools that
|
||||
// dereference an admin-owned connection rather than a caller-supplied host — see
|
||||
// ApiConnectionResolver.
|
||||
private static final Pattern ALLOWED_ENDPOINT_PATH =
|
||||
Pattern.compile(
|
||||
"^/api/v1/(general|misc|security|convert|filter)(/[A-Za-z0-9_-]+)+$"
|
||||
"^/api/v1/(general|misc|security|convert|filter|integration|docparse)(/[A-Za-z0-9_-]+)+$"
|
||||
+ "|^/api/v1/ai/tools(/[A-Za-z0-9_-]+)+$");
|
||||
|
||||
/**
|
||||
|
||||
@@ -230,6 +230,18 @@ public class TaskManager {
|
||||
return false;
|
||||
}
|
||||
|
||||
/** Attach metadata to a job and write it through to the shared store for cluster peers. */
|
||||
public boolean putMetadata(String jobId, String key, String value) {
|
||||
JobResult jobResult = jobResults.get(jobId);
|
||||
if (jobResult != null) {
|
||||
jobResult.putMetadata(key, value);
|
||||
writeThrough(jobId, jobResult);
|
||||
return true;
|
||||
}
|
||||
log.warn("Attempted to set metadata on non-existent job ID: {}", jobId);
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get statistics about all jobs in the system
|
||||
*
|
||||
@@ -378,7 +390,7 @@ public class TaskManager {
|
||||
fileIds.add(rf.getFileId());
|
||||
}
|
||||
}
|
||||
Map<String, String> meta = new HashMap<>();
|
||||
Map<String, String> meta = new HashMap<>(result.getMetadata());
|
||||
if (result.getNotes() != null && !result.getNotes().isEmpty()) {
|
||||
meta.put("notesCount", Integer.toString(result.getNotes().size()));
|
||||
}
|
||||
|
||||
@@ -5,6 +5,7 @@ import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Locale;
|
||||
import java.util.Optional;
|
||||
import java.util.regex.Pattern;
|
||||
|
||||
import org.apache.pdfbox.pdmodel.PDDocument;
|
||||
import org.apache.pdfbox.pdmodel.common.PDRectangle;
|
||||
@@ -28,6 +29,8 @@ import lombok.extern.slf4j.Slf4j;
|
||||
@Component
|
||||
public class PdfTextLocator {
|
||||
|
||||
private static final Pattern NON_ALPHANUMERIC_PATTERN = Pattern.compile("[^A-Za-z0-9]");
|
||||
|
||||
/** One found line of text with its user-space bounding box. */
|
||||
public record MatchedBox(float x, float y, float width, float height) {}
|
||||
|
||||
@@ -82,7 +85,7 @@ public class PdfTextLocator {
|
||||
|
||||
/** Strip everything non-alphanumeric and lowercase for tolerant matching. */
|
||||
private static String normalize(String s) {
|
||||
return s.replaceAll("[^A-Za-z0-9]", "").toLowerCase(Locale.ROOT);
|
||||
return NON_ALPHANUMERIC_PATTERN.matcher(s).replaceAll("").toLowerCase(Locale.ROOT);
|
||||
}
|
||||
|
||||
private static final class CapturedLine {
|
||||
|
||||
@@ -1,7 +1,11 @@
|
||||
package stirling.software.common.util;
|
||||
|
||||
import java.util.regex.Pattern;
|
||||
|
||||
public class RequestUriUtils {
|
||||
|
||||
private static final Pattern SHARE_LINK_PATTERN = Pattern.compile("^/share/[^/]+/?$");
|
||||
|
||||
public static boolean isStaticResource(String requestURI) {
|
||||
return isStaticResource("", requestURI);
|
||||
}
|
||||
@@ -198,11 +202,12 @@ public class RequestUriUtils {
|
||||
|| trimmedUri.startsWith("/readiness")
|
||||
|| trimmedUri.startsWith(
|
||||
"/api/v1/mobile-scanner/") // Mobile scanner endpoints (no auth)
|
||||
|| trimmedUri.startsWith("/api/v1/webhooks/")
|
||||
|| trimmedUri.startsWith("/v1/api-docs")
|
||||
// Workflow participant endpoints - access controlled by share tokens, not login
|
||||
|| trimmedUri.startsWith("/api/v1/workflow/participant/")
|
||||
// Share-link SPA bootstrap; data APIs remain protected
|
||||
|| trimmedUri.matches("^/share/[^/]+/?$");
|
||||
|| SHARE_LINK_PATTERN.matcher(trimmedUri).matches();
|
||||
}
|
||||
|
||||
private static String stripContextPath(String contextPath, String requestURI) {
|
||||
|
||||
@@ -176,6 +176,13 @@ class RequestUriUtilsTest {
|
||||
assertFalse(RequestUriUtils.isPublicAuthEndpoint("/api/v1/convert", ""));
|
||||
}
|
||||
|
||||
@Test
|
||||
void testIsPublicAuthEndpoint_webhookReceiver() {
|
||||
// The webhook source receiver authenticates each delivery by HMAC signature, not a session.
|
||||
assertTrue(RequestUriUtils.isPublicAuthEndpoint("/api/v1/webhooks/whk_abc123", ""));
|
||||
assertTrue(RequestUriUtils.isPublicAuthEndpoint("/app/api/v1/webhooks/whk_abc123", "/app"));
|
||||
}
|
||||
|
||||
@Test
|
||||
void testIsPublicAuthEndpoint_withContextPath() {
|
||||
assertTrue(RequestUriUtils.isPublicAuthEndpoint("/app/login", "/app"));
|
||||
|
||||
@@ -9,35 +9,6 @@ configurations {
|
||||
}
|
||||
}
|
||||
|
||||
spotless {
|
||||
java {
|
||||
target 'src/**/java/**/*.java'
|
||||
targetExclude 'src/main/resources/static/**', 'src/main/java/org/apache/**'
|
||||
googleJavaFormat(googleJavaFormatVersion).aosp().reorderImports(false)
|
||||
// google-java-format 1.28.0 bundles Guava 32.x which crashes Spotless lint on JDK 24/25
|
||||
suppressLintsFor { setStep('google-java-format') }
|
||||
|
||||
importOrder("java", "javax", "org", "com", "net", "io", "jakarta", "lombok", "me", "stirling")
|
||||
trimTrailingWhitespace()
|
||||
leadingTabsToSpaces()
|
||||
endWithNewline()
|
||||
}
|
||||
yaml {
|
||||
target '**/*.yml', '**/*.yaml'
|
||||
targetExclude 'src/main/resources/static/**'
|
||||
trimTrailingWhitespace()
|
||||
leadingTabsToSpaces()
|
||||
endWithNewline()
|
||||
}
|
||||
format 'gradle', {
|
||||
target '**/gradle/*.gradle', '**/*.gradle'
|
||||
targetExclude 'src/main/resources/static/**'
|
||||
trimTrailingWhitespace()
|
||||
leadingTabsToSpaces()
|
||||
endWithNewline()
|
||||
}
|
||||
}
|
||||
|
||||
dependencies {
|
||||
if (!gradle.ext.disableAdditional) {
|
||||
implementation project(':proprietary')
|
||||
|
||||
+8
-8
@@ -117,8 +117,8 @@ final class FormPayloadParser {
|
||||
names.add(single);
|
||||
}
|
||||
}
|
||||
} else if (root.isTextual()) {
|
||||
final String single = trimToNull(root.asText(""));
|
||||
} else if (root.isString()) {
|
||||
final String single = trimToNull(root.asString(""));
|
||||
if (single != null) {
|
||||
names.add(single);
|
||||
}
|
||||
@@ -197,8 +197,8 @@ final class FormPayloadParser {
|
||||
if (node == null || node.isNull()) {
|
||||
return null;
|
||||
}
|
||||
if (node.isTextual()) {
|
||||
return trimToEmpty(node.asText(""));
|
||||
if (node.isString()) {
|
||||
return trimToEmpty(node.asString(""));
|
||||
}
|
||||
if (node.isNumber()) {
|
||||
return node.numberValue().toString();
|
||||
@@ -207,7 +207,7 @@ final class FormPayloadParser {
|
||||
return Boolean.toString(node.booleanValue());
|
||||
}
|
||||
// Fallback for other scalar-like nodes
|
||||
return trimToEmpty(node.asText(""));
|
||||
return trimToEmpty(node.asString(""));
|
||||
}
|
||||
|
||||
private static void collectNames(JsonNode arrayNode, Set<String> sink) {
|
||||
@@ -227,8 +227,8 @@ final class FormPayloadParser {
|
||||
return null;
|
||||
}
|
||||
|
||||
if (node.isTextual()) {
|
||||
return trimToNull(node.asText(""));
|
||||
if (node.isString()) {
|
||||
return trimToNull(node.asString(""));
|
||||
}
|
||||
|
||||
if (node.isObject()) {
|
||||
@@ -269,7 +269,7 @@ final class FormPayloadParser {
|
||||
final JsonNode v = objectNode.get(key);
|
||||
if (v == null || v.isNull()) {
|
||||
result.put(key, null);
|
||||
} else if (v.isTextual() || v.isNumber() || v.isBoolean()) {
|
||||
} else if (v.isString() || v.isNumber() || v.isBoolean()) {
|
||||
result.put(key, coerceScalarToString(v));
|
||||
} else {
|
||||
result.put(key, v.toString());
|
||||
|
||||
+29
-2
@@ -24,6 +24,7 @@ import stirling.software.common.annotations.api.ConfigApi;
|
||||
import stirling.software.common.configuration.AppConfig;
|
||||
import stirling.software.common.configuration.interfaces.ShowAdminInterface;
|
||||
import stirling.software.common.model.ApplicationProperties;
|
||||
import stirling.software.common.service.DocparseCapabilityServiceInterface;
|
||||
import stirling.software.common.service.ServerCertificateServiceInterface;
|
||||
import stirling.software.common.service.UserServiceInterface;
|
||||
import stirling.software.common.util.GeneralUtils;
|
||||
@@ -41,6 +42,7 @@ public class ConfigController {
|
||||
private final ShowAdminInterface showAdmin;
|
||||
private final stirling.software.common.service.LicenseServiceInterface licenseService;
|
||||
private final stirling.software.SPDF.config.ExternalAppDepConfig externalAppDepConfig;
|
||||
private final DocparseCapabilityServiceInterface docparseCapabilityService;
|
||||
|
||||
public ConfigController(
|
||||
ApplicationProperties applicationProperties,
|
||||
@@ -54,7 +56,9 @@ public class ConfigController {
|
||||
ShowAdminInterface showAdmin,
|
||||
@org.springframework.beans.factory.annotation.Autowired(required = false)
|
||||
stirling.software.common.service.LicenseServiceInterface licenseService,
|
||||
stirling.software.SPDF.config.ExternalAppDepConfig externalAppDepConfig) {
|
||||
stirling.software.SPDF.config.ExternalAppDepConfig externalAppDepConfig,
|
||||
@org.springframework.beans.factory.annotation.Autowired(required = false)
|
||||
DocparseCapabilityServiceInterface docparseCapabilityService) {
|
||||
this.applicationProperties = applicationProperties;
|
||||
this.applicationContext = applicationContext;
|
||||
this.endpointConfiguration = endpointConfiguration;
|
||||
@@ -63,6 +67,7 @@ public class ConfigController {
|
||||
this.showAdmin = showAdmin;
|
||||
this.licenseService = licenseService;
|
||||
this.externalAppDepConfig = externalAppDepConfig;
|
||||
this.docparseCapabilityService = docparseCapabilityService;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -336,7 +341,29 @@ public class ConfigController {
|
||||
configData.put("premiumEnabled", applicationProperties.getPremium().isEnabled());
|
||||
|
||||
// AI Engine settings
|
||||
configData.put("aiEngineEnabled", applicationProperties.getAiEngine().isEnabled());
|
||||
ApplicationProperties.AiEngine aiEngineConfig = applicationProperties.getAiEngine();
|
||||
configData.put("aiEngineEnabled", aiEngineConfig.isEnabled());
|
||||
// Per-capability flags let the UI hide individual AI tools an admin has turned off.
|
||||
ApplicationProperties.AiEngine.Features aiFeatures = aiEngineConfig.getFeatures();
|
||||
configData.put(
|
||||
"aiFeatures",
|
||||
Map.ofEntries(
|
||||
Map.entry("chat", aiFeatures.isChat()),
|
||||
Map.entry("documentQuestions", aiFeatures.isDocumentQuestions()),
|
||||
Map.entry("createPdf", aiFeatures.isCreatePdf()),
|
||||
Map.entry("mathAuditor", aiFeatures.isMathAuditor()),
|
||||
Map.entry("pdfComment", aiFeatures.isPdfComment()),
|
||||
Map.entry("classify", aiFeatures.isClassify())));
|
||||
|
||||
// DocParse settings; "advanced" reflects the cached engine capability probe and is
|
||||
// false when the engine is disabled, unreachable, or the proprietary module is absent.
|
||||
boolean docparseEnabled = applicationProperties.getDocparse().isEnabled();
|
||||
configData.put("docparseEnabled", docparseEnabled);
|
||||
configData.put(
|
||||
"docparseAdvanced",
|
||||
docparseEnabled
|
||||
&& docparseCapabilityService != null
|
||||
&& docparseCapabilityService.isAdvancedInstalled());
|
||||
|
||||
// Timestamp TSA settings — single source of truth for presets + admin URLs
|
||||
ApplicationProperties.Security.Timestamp tsConfig =
|
||||
|
||||
+66
-6
@@ -23,6 +23,9 @@ import org.bouncycastle.cms.CMSSignedData;
|
||||
import org.bouncycastle.cms.SignerInformation;
|
||||
import org.bouncycastle.cms.SignerInformationStore;
|
||||
import org.bouncycastle.cms.jcajce.JcaSimpleSignerInfoVerifierBuilder;
|
||||
import org.bouncycastle.operator.jcajce.JcaDigestCalculatorProviderBuilder;
|
||||
import org.bouncycastle.tsp.TimeStampToken;
|
||||
import org.bouncycastle.tsp.TimeStampTokenInfo;
|
||||
import org.bouncycastle.util.Store;
|
||||
import org.springframework.http.MediaType;
|
||||
import org.springframework.http.ResponseEntity;
|
||||
@@ -54,6 +57,9 @@ public class ValidateSignatureController {
|
||||
private final CustomPDFDocumentFactory pdfDocumentFactory;
|
||||
private final CertificateValidationService certValidationService;
|
||||
|
||||
/** PDF sub-filter identifying an RFC 3161 document timestamp (PAdES-LTV). */
|
||||
private static final String SUBFILTER_RFC3161 = "ETSI.RFC3161";
|
||||
|
||||
@InitBinder
|
||||
public void initBinder(WebDataBinder binder) {
|
||||
binder.registerCustomEditor(
|
||||
@@ -128,8 +134,35 @@ public class ValidateSignatureController {
|
||||
byte[] signedContent = sig.getSignedContent(file.getInputStream());
|
||||
byte[] signatureBytes = sig.getContents(file.getInputStream());
|
||||
|
||||
CMSProcessable content = new CMSProcessableByteArray(signedContent);
|
||||
CMSSignedData signedData = new CMSSignedData(content, signatureBytes);
|
||||
// An RFC 3161 document timestamp (PAdES-LTV) carries its signed content
|
||||
// *inside* the CMS - a TSTInfo - rather than being detached over the document.
|
||||
// Building it as detached digests the ByteRange against an attribute that
|
||||
// covers the TSTInfo, which can never match.
|
||||
boolean isDocTimeStamp = SUBFILTER_RFC3161.equals(sig.getSubFilter());
|
||||
CMSSignedData signedData;
|
||||
if (isDocTimeStamp) {
|
||||
signedData = new CMSSignedData(signatureBytes);
|
||||
} else {
|
||||
CMSProcessable content = new CMSProcessableByteArray(signedContent);
|
||||
signedData = new CMSSignedData(content, signatureBytes);
|
||||
}
|
||||
|
||||
// What actually binds a timestamp to this document: the TSTInfo's message
|
||||
// imprint must equal the digest of the signed byte range. Without this check a
|
||||
// valid timestamp token for some *other* document would verify happily here.
|
||||
Date timeStampGenTime = null;
|
||||
if (isDocTimeStamp) {
|
||||
TimeStampToken token = new TimeStampToken(signedData);
|
||||
TimeStampTokenInfo info = token.getTimeStampInfo();
|
||||
timeStampGenTime = info.getGenTime();
|
||||
if (!timestampCoversContent(info, signedContent)) {
|
||||
result.setValid(false);
|
||||
result.setErrorMessage(
|
||||
"Timestamp message imprint does not match the document");
|
||||
results.add(result);
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
Store<X509CertificateHolder> certStore = signedData.getCertificates();
|
||||
SignerInformationStore signerStore = signedData.getSignerInfos();
|
||||
@@ -162,7 +195,15 @@ public class ValidateSignatureController {
|
||||
CertificateValidationService.ValidationTime validationTimeResult =
|
||||
certValidationService.extractValidationTime(signerInfo);
|
||||
Date validationTime;
|
||||
if (validationTimeResult == null) {
|
||||
if (timeStampGenTime != null) {
|
||||
// The TSA's own asserted time is the authoritative one here, and is
|
||||
// exactly what makes the signature verifiable after the cert expires.
|
||||
validationTime = timeStampGenTime;
|
||||
// Distinct from "timestamp", which CertificateValidationService already
|
||||
// uses for a signature countersigned by a TSA. Both are RFC 3161, but
|
||||
// one attests a signature and the other attests the whole document.
|
||||
result.setValidationTimeSource("document-timestamp");
|
||||
} else if (validationTimeResult == null) {
|
||||
validationTime = new Date();
|
||||
result.setValidationTimeSource("current");
|
||||
} else {
|
||||
@@ -235,10 +276,13 @@ public class ValidateSignatureController {
|
||||
|
||||
// Set basic signature info
|
||||
result.setSignerName(sig.getName());
|
||||
// A DocTimeStamp has no /M entry; its date is the TSA's genTime.
|
||||
result.setSignatureDate(
|
||||
sig.getSignDate() != null
|
||||
? sig.getSignDate().getTime().toString()
|
||||
: null);
|
||||
timeStampGenTime != null
|
||||
? timeStampGenTime.toString()
|
||||
: sig.getSignDate() != null
|
||||
? sig.getSignDate().getTime().toString()
|
||||
: null);
|
||||
result.setReason(sig.getReason());
|
||||
result.setLocation(sig.getLocation());
|
||||
|
||||
@@ -301,4 +345,20 @@ public class ValidateSignatureController {
|
||||
|
||||
return ResponseEntity.ok(results);
|
||||
}
|
||||
|
||||
/**
|
||||
* True when the timestamp token was issued over exactly these bytes.
|
||||
*
|
||||
* <p>The digest algorithm is taken from the token rather than assumed, because a TSA chooses it
|
||||
* - assuming SHA-256 would silently fail against any TSA that uses something else.
|
||||
*/
|
||||
private static boolean timestampCoversContent(TimeStampTokenInfo info, byte[] signedContent)
|
||||
throws Exception {
|
||||
org.bouncycastle.operator.DigestCalculator digest =
|
||||
new JcaDigestCalculatorProviderBuilder().build().get(info.getHashAlgorithm());
|
||||
try (java.io.OutputStream out = digest.getOutputStream()) {
|
||||
out.write(signedContent);
|
||||
}
|
||||
return java.util.Arrays.equals(digest.getDigest(), info.getMessageImprintDigest());
|
||||
}
|
||||
}
|
||||
|
||||
@@ -18,10 +18,10 @@ public class ApiEndpoint {
|
||||
postNode.path("parameters")
|
||||
.forEach(
|
||||
paramNode -> {
|
||||
String paramName = paramNode.path("name").asText("");
|
||||
String paramName = paramNode.path("name").asString("");
|
||||
parameters.put(paramName, paramNode);
|
||||
});
|
||||
this.description = postNode.path("description").asText("");
|
||||
this.description = postNode.path("description").asString("");
|
||||
}
|
||||
|
||||
public boolean areParametersValid(Map<String, Object> providedParams) {
|
||||
|
||||
@@ -72,6 +72,8 @@ spring.datasource.username=sa
|
||||
spring.datasource.password=
|
||||
spring.h2.console.enabled=false
|
||||
spring.jpa.hibernate.ddl-auto=update
|
||||
# Batch associations into IN() loads so list endpoints don't N+1 as tables grow.
|
||||
spring.jpa.properties.hibernate.default_batch_fetch_size=100
|
||||
# Defer datasource initialization to ensure that the database is fully set up
|
||||
# before Hibernate attempts to access it. This is particularly useful when
|
||||
# using database initialization scripts or tools.
|
||||
@@ -96,7 +98,8 @@ spring.main.allow-bean-definition-overriding=true
|
||||
# spring-data-redis is on the classpath only for the optional Valkey backplane (which wires its own
|
||||
# factory); exclude Spring Boot's stock Redis auto-config so a default install doesn't create a dead
|
||||
# localhost:6379 factory that flips /actuator/health to DOWN.
|
||||
spring.autoconfigure.exclude=org.springframework.boot.data.redis.autoconfigure.DataRedisAutoConfiguration,org.springframework.boot.data.redis.autoconfigure.DataRedisReactiveAutoConfiguration
|
||||
# Also exclude the repositories auto-config: in cluster mode it needs a redisTemplate bean we don't define.
|
||||
spring.autoconfigure.exclude=org.springframework.boot.data.redis.autoconfigure.DataRedisAutoConfiguration,org.springframework.boot.data.redis.autoconfigure.DataRedisReactiveAutoConfiguration,org.springframework.boot.data.redis.autoconfigure.DataRedisRepositoriesAutoConfiguration
|
||||
|
||||
# Set up a consistent temporary directory location
|
||||
java.io.tmpdir=${stirling.tempfiles.directory:${java.io.tmpdir}/stirling-pdf}
|
||||
|
||||
@@ -366,12 +366,51 @@ aiEngine:
|
||||
enabled: false # Set to 'true' to enable the AI engine integration
|
||||
url: http://localhost:5001 # URL of the Python AI engine
|
||||
timeoutSeconds: 120 # Timeout in seconds for AI engine requests
|
||||
longRunningTimeoutSeconds: 600 # Timeout (seconds) for heavy operations like RAG ingestion of large documents
|
||||
streamTimeoutSeconds: 1800 # SSE stream timeout (seconds) for long-running orchestrator runs
|
||||
pushConfigToEngine: true # Push admin AI config to the engine on startup + save; false = engine stays fully env-controlled
|
||||
models:
|
||||
provider: anthropic # Model provider: 'anthropic', 'openai', 'ollama', or 'custom' (OpenAI-compatible)
|
||||
smartModel: claude-haiku-4-5 # High-quality tier model name (no provider prefix)
|
||||
fastModel: claude-haiku-4-5 # Cheap/fast tier model name (no provider prefix)
|
||||
smartMaxTokens: 8192 # Max output tokens for the smart tier
|
||||
fastMaxTokens: 2048 # Max output tokens for the fast tier
|
||||
apiKey: "" # API key for the selected provider (secret). Empty = engine uses its native env credentials (e.g. ANTHROPIC_API_KEY)
|
||||
baseUrl: "" # OpenAI-compatible base URL for 'ollama'/'custom' providers (e.g. http://ollama:11434/v1). Ignored for anthropic/openai
|
||||
rag:
|
||||
embeddingProvider: voyageai # Embedding provider: 'voyageai', 'openai', 'ollama', or 'custom' (OpenAI-compatible)
|
||||
embeddingModel: voyage-4 # Embedding model name (no provider prefix)
|
||||
embeddingApiKey: "" # Secret API key for the embedding provider. Empty = engine uses its native env credentials (e.g. VOYAGE_API_KEY)
|
||||
embeddingBaseUrl: "" # OpenAI-compatible base URL for 'ollama'/'custom' embedding providers (e.g. http://ollama:11434/v1). Ignored for voyageai/openai
|
||||
topK: 20 # Number of chunks retrieval returns per search
|
||||
maxSearches: 5 # Per-run cap on knowledge-search tool calls before the agent must answer
|
||||
limits:
|
||||
maxPages: 200 # Upper bound on PDF pages the engine will process per request
|
||||
maxCharacters: 200000 # Upper bound on characters of extracted text per request
|
||||
modelMaxConcurrency: 32 # Process-wide cap on concurrent model API calls (engine restart to apply)
|
||||
features: # Per-capability switches; turn an individual AI tool off without disabling the whole engine
|
||||
chat: true # Assistant chat
|
||||
documentQuestions: true # Ask-questions-about-a-PDF
|
||||
createPdf: true # Generate a PDF from a natural-language spec
|
||||
mathAuditor: true # Numerical/formula contradiction auditing
|
||||
pdfComment: true # AI-authored PDF comments/annotations
|
||||
classify: true # Automatic document classification/labelling
|
||||
|
||||
# DocParse: document understanding for ingestion pipelines (chunking + knowledge-base
|
||||
# indexing). The basic tier (text layer) always works; the advanced tier (layout parsing)
|
||||
# requires the engine's docparse addon. Env overrides: DOCPARSE_ENABLED, DOCPARSE_MODE.
|
||||
docparse:
|
||||
enabled: true # Master switch; hides the DocParse endpoints when false
|
||||
mode: auto # Tier selection: 'auto' (best available), 'basic', or 'advanced'
|
||||
autoInstall: false # Mirrors DOCPARSE_AUTO_INSTALL for the engine's boot-time addon install script
|
||||
|
||||
policies:
|
||||
# Folder automations can read from and write to the directories you allow here, so treat this as a
|
||||
# security boundary. Leave allowedFolderRoots empty (default) to disable folder sources/outputs
|
||||
# entirely; list absolute directories to permit folder access only within them. Stirling's own
|
||||
# config directory is always off-limits, and folder access is always disabled in SaaS mode.
|
||||
# security boundary. Leave allowedFolderRoots empty (default) to disable folder sources/outputs,
|
||||
# other than from directories that are always permitted like server file-storage and watched folders.
|
||||
# List absolute directories to permit folder access within them.
|
||||
# Stirling's own config directory is always off-limits, and folder access is always
|
||||
# disabled in SaaS mode.
|
||||
allowedFolderRoots: [] # e.g. ["/data/inbox", "/data/outbox"]
|
||||
scheduleSweepSeconds: 60 # How often (seconds) scheduled policies are checked for being due
|
||||
watchReconcileSeconds: 300 # How often (seconds) folder-watch re-syncs watches and re-runs as a safety net for missed events
|
||||
@@ -385,6 +424,8 @@ policies:
|
||||
mcp:
|
||||
enabled: false # Master switch. 'false' (default) means no /mcp endpoint, no metadata, no beans wired.
|
||||
scopesEnabled: true # Enforce mcp.tools.read / mcp.tools.write scopes derived from operation category
|
||||
maxRequestBytes: 10485760 # Max size (bytes) of an incoming MCP tool request payload (default 10 MB)
|
||||
maxInlineResponseBytes: 10485760 # Max size (bytes) of an MCP tool response returned inline before it is rejected (default 10 MB)
|
||||
allowedOperations: [] # Tool allow-list (operation ids, e.g. ['compress-pdf']). Empty = all. When set, ONLY these are exposed over MCP.
|
||||
blockedOperations: [] # Tool deny-list (operation ids). Always removed from MCP even if otherwise allowed.
|
||||
auth:
|
||||
|
||||
+2
-1
@@ -74,7 +74,8 @@ class ConfigControllerMoreTest {
|
||||
userService,
|
||||
showAdmin,
|
||||
licenseService,
|
||||
externalAppDepConfig);
|
||||
externalAppDepConfig,
|
||||
null);
|
||||
}
|
||||
|
||||
@SuppressWarnings("unchecked")
|
||||
|
||||
+2
-1
@@ -52,7 +52,8 @@ class ConfigControllerTest {
|
||||
userService,
|
||||
showAdmin,
|
||||
licenseService,
|
||||
mock(stirling.software.SPDF.config.ExternalAppDepConfig.class));
|
||||
mock(stirling.software.SPDF.config.ExternalAppDepConfig.class),
|
||||
null);
|
||||
}
|
||||
|
||||
@Test
|
||||
|
||||
+91
@@ -0,0 +1,91 @@
|
||||
package stirling.software.SPDF.controller.api.security;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.mockito.ArgumentMatchers.any;
|
||||
import static org.mockito.Mockito.when;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.io.InputStream;
|
||||
import java.util.List;
|
||||
|
||||
import org.apache.pdfbox.Loader;
|
||||
import org.junit.jupiter.api.BeforeEach;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.springframework.core.io.ClassPathResource;
|
||||
import org.springframework.mock.web.MockMultipartFile;
|
||||
|
||||
import stirling.software.SPDF.model.api.security.SignatureValidationRequest;
|
||||
import stirling.software.SPDF.model.api.security.SignatureValidationResult;
|
||||
import stirling.software.SPDF.service.CertificateValidationService;
|
||||
import stirling.software.common.model.ApplicationProperties;
|
||||
import stirling.software.common.service.CustomPDFDocumentFactory;
|
||||
|
||||
/**
|
||||
* Validation of RFC 3161 document timestamps (PAdES-LTV).
|
||||
*
|
||||
* <p>These fixtures are a real PDF stamped by a real public TSA (freetsa.org). Before this was
|
||||
* handled explicitly, every such timestamp was reported invalid: a DocTimeStamp's CMS encapsulates
|
||||
* a TSTInfo rather than being detached over the document, so digesting the byte range compared
|
||||
* against the wrong thing and always mismatched. That made the timestamp feature look broken to
|
||||
* anyone who checked their own output with our validator.
|
||||
*/
|
||||
class DocumentTimestampValidationTest {
|
||||
|
||||
private ValidateSignatureController controller;
|
||||
|
||||
@BeforeEach
|
||||
void setUp() throws Exception {
|
||||
CertificateValidationService certValidationService =
|
||||
new CertificateValidationService(null, new ApplicationProperties());
|
||||
CustomPDFDocumentFactory factory = org.mockito.Mockito.mock(CustomPDFDocumentFactory.class);
|
||||
// Delegate to the real loader so the signature dictionary is parsed as in production.
|
||||
when(factory.load(any(InputStream.class)))
|
||||
.thenAnswer(
|
||||
invocation ->
|
||||
Loader.loadPDF(
|
||||
((InputStream) invocation.getArgument(0)).readAllBytes()));
|
||||
controller = new ValidateSignatureController(factory, certValidationService);
|
||||
}
|
||||
|
||||
@Test
|
||||
void aGenuineDocumentTimestampValidates() throws Exception {
|
||||
SignatureValidationResult result = validate("timestamp/doc-timestamped.pdf");
|
||||
|
||||
assertThat(result.isValid()).isTrue();
|
||||
assertThat(result.getErrorMessage()).isNull();
|
||||
// The TSA's asserted time is what keeps the signature verifiable once the signing
|
||||
// certificate expires, so it must be the time we validate against.
|
||||
// Deliberately not "timestamp" - that value already means "signature countersigned by a
|
||||
// TSA", which is a different assertion about a different thing.
|
||||
assertThat(result.getValidationTimeSource()).isEqualTo("document-timestamp");
|
||||
assertThat(result.getSignatureDate()).isNotNull();
|
||||
assertThat(result.getSubjectDN()).contains("freetsa.org");
|
||||
assertThat(result.isCoversEntireDocument()).isTrue();
|
||||
}
|
||||
|
||||
@Test
|
||||
void aTamperedDocumentFailsTheMessageImprintCheck() throws Exception {
|
||||
// Same file with a single byte flipped inside the signed range. Without the imprint check
|
||||
// the CMS signature over the TSTInfo would still verify happily - the token is untouched -
|
||||
// and a modified document would be reported as validly timestamped.
|
||||
SignatureValidationResult result = validate("timestamp/doc-timestamped-tampered.pdf");
|
||||
|
||||
assertThat(result.isValid()).isFalse();
|
||||
assertThat(result.getErrorMessage())
|
||||
.isEqualTo("Timestamp message imprint does not match the document");
|
||||
}
|
||||
|
||||
private SignatureValidationResult validate(String resource) throws IOException {
|
||||
byte[] bytes;
|
||||
try (InputStream in = new ClassPathResource(resource).getInputStream()) {
|
||||
bytes = in.readAllBytes();
|
||||
}
|
||||
SignatureValidationRequest request = new SignatureValidationRequest();
|
||||
request.setFileInput(
|
||||
new MockMultipartFile("fileInput", "doc.pdf", "application/pdf", bytes));
|
||||
|
||||
List<SignatureValidationResult> results = controller.validateSignature(request).getBody();
|
||||
assertThat(results).hasSize(1);
|
||||
return results.get(0);
|
||||
}
|
||||
}
|
||||
Binary file not shown.
Binary file not shown.
@@ -6,33 +6,6 @@ repositories {
|
||||
bootRun {
|
||||
enabled = false
|
||||
}
|
||||
|
||||
spotless {
|
||||
java {
|
||||
target 'src/**/java/**/*.java'
|
||||
targetExclude 'src/main/java/org/apache/**'
|
||||
googleJavaFormat(googleJavaFormatVersion).aosp().reorderImports(false)
|
||||
// google-java-format 1.28.0 bundles Guava 32.x which crashes Spotless lint on JDK 24/25
|
||||
suppressLintsFor { setStep('google-java-format') }
|
||||
|
||||
importOrder("java", "javax", "org", "com", "net", "io", "jakarta", "lombok", "me", "stirling")
|
||||
trimTrailingWhitespace()
|
||||
leadingTabsToSpaces()
|
||||
endWithNewline()
|
||||
}
|
||||
yaml {
|
||||
target '**/*.yml', '**/*.yaml'
|
||||
trimTrailingWhitespace()
|
||||
leadingTabsToSpaces()
|
||||
endWithNewline()
|
||||
}
|
||||
format 'gradle', {
|
||||
target '**/gradle/*.gradle', '**/*.gradle'
|
||||
trimTrailingWhitespace()
|
||||
leadingTabsToSpaces()
|
||||
endWithNewline()
|
||||
}
|
||||
}
|
||||
dependencies {
|
||||
implementation project(':common')
|
||||
api "com.google.guava:guava:${guavaVersion}"
|
||||
@@ -103,6 +76,7 @@ dependencies {
|
||||
testImplementation "org.testcontainers:testcontainers:${testcontainersMinioVersion}"
|
||||
testImplementation "org.testcontainers:minio:${testcontainersMinioVersion}"
|
||||
testImplementation "org.testcontainers:localstack:${testcontainersMinioVersion}"
|
||||
testImplementation "org.testcontainers:postgresql:${testcontainersMinioVersion}"
|
||||
testImplementation "org.testcontainers:junit-jupiter:${testcontainersMinioVersion}"
|
||||
}
|
||||
|
||||
|
||||
+52
-3
@@ -1,5 +1,6 @@
|
||||
package stirling.software.proprietary.access.service;
|
||||
|
||||
import java.util.Collection;
|
||||
import java.util.HashSet;
|
||||
import java.util.List;
|
||||
import java.util.Set;
|
||||
@@ -42,6 +43,52 @@ public class ResourceAccessService {
|
||||
return canUseResource(ResourceType.PORTAL, "", null, portalDefaultPolicy, user);
|
||||
}
|
||||
|
||||
/**
|
||||
* Portal access for a roster (admin, grant, or default policy). {@code activeTeamLeaderUserIds}
|
||||
* must hold ids of users who lead their own active team — the set the ADMINS_AND_TEAM_LEADS
|
||||
* default admits, matching {@link #canAccessPortal}.
|
||||
*/
|
||||
public Set<Long> usersWithPortalAccess(
|
||||
Collection<User> users, Set<Long> activeTeamLeaderUserIds) {
|
||||
Set<PrincipalRef> grantedPrincipals = new HashSet<>();
|
||||
for (ResourceGrant g :
|
||||
grantRepository.findByResourceTypeAndResourceId(ResourceType.PORTAL, "")) {
|
||||
if (permissionSatisfies(g.getPermission(), AccessPermission.USE)) {
|
||||
grantedPrincipals.add(new PrincipalRef(g.getPrincipalType(), g.getPrincipalId()));
|
||||
}
|
||||
}
|
||||
Set<Long> leaderIds = activeTeamLeaderUserIds == null ? Set.of() : activeTeamLeaderUserIds;
|
||||
Set<Long> allowed = new HashSet<>();
|
||||
for (User user : users) {
|
||||
if (user != null
|
||||
&& user.getId() != null
|
||||
&& hasPortalAccess(user, grantedPrincipals, leaderIds)) {
|
||||
allowed.add(user.getId());
|
||||
}
|
||||
}
|
||||
return allowed;
|
||||
}
|
||||
|
||||
private boolean hasPortalAccess(
|
||||
User user, Set<PrincipalRef> grantedPrincipals, Set<Long> leaderIds) {
|
||||
if (isAdmin(user)) {
|
||||
return true;
|
||||
}
|
||||
for (PrincipalRef principal : principalResolver.principalsOf(user)) {
|
||||
if (grantedPrincipals.contains(principal)) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
if (portalDefaultPolicy == null) {
|
||||
return false;
|
||||
}
|
||||
return switch (portalDefaultPolicy) {
|
||||
case ORG_ALL -> principalResolver.allowsDeploymentWideAccess();
|
||||
case ADMINS_AND_TEAM_LEADS -> leaderIds.contains(user.getId());
|
||||
case EXPLICIT_ONLY -> false;
|
||||
};
|
||||
}
|
||||
|
||||
/** Whether the user may use a resource, falling back to its default policy. */
|
||||
public boolean canUseResource(
|
||||
ResourceType type,
|
||||
@@ -172,11 +219,13 @@ public class ResourceAccessService {
|
||||
};
|
||||
}
|
||||
|
||||
// Portal (no owner) admits any team lead; a team-owned resource admits only that team's
|
||||
// leads; a user-owned resource admits no extra leads.
|
||||
// Portal (no owner) admits the leader of the user's active team; a team-owned resource
|
||||
// admits only that team's leads; a user-owned resource admits no extra leads.
|
||||
private boolean matchesTeamLeadDefault(PrincipalRef owner, User user) {
|
||||
if (owner == null) {
|
||||
return teamLeadLookup.isAnyTeamLeader(user);
|
||||
return user.getTeam() != null
|
||||
&& user.getTeam().getId() != null
|
||||
&& teamLeadLookup.isLeaderOfTeam(user, user.getTeam().getId());
|
||||
}
|
||||
return owner.type() == PrincipalType.TEAM
|
||||
&& owner.id() != null
|
||||
|
||||
+69
@@ -39,6 +39,11 @@ public class SecretMasker {
|
||||
"bearer",
|
||||
"signature");
|
||||
|
||||
// Keys whose nested map holds secrets under arbitrary, caller-named keys - a free-form HTTP
|
||||
// headers map is the case in point: the secret can sit under any header name (X-API-Key,
|
||||
// Ocp-Apim-Subscription-Key), so the name is no signal. Mask every value in these outright.
|
||||
private static final Set<String> SENSITIVE_VALUE_CONTAINERS = Set.of("headers");
|
||||
|
||||
/** Replace sensitive values with the mask (recursively) for safe display. */
|
||||
public Map<String, Object> mask(Map<String, Object> config) {
|
||||
return mask(config, 0);
|
||||
@@ -73,6 +78,12 @@ public class SecretMasker {
|
||||
if (isSensitive(e.getKey()) && isRedacted(e.getValue(), depth)) {
|
||||
continue;
|
||||
}
|
||||
if (isSensitiveContainer(e.getKey())
|
||||
&& e.getValue() instanceof Map<?, ?> m
|
||||
&& depth < MAX_DEPTH) {
|
||||
out.put(e.getKey(), sanitizeAllValues(castMap(m), depth + 1));
|
||||
continue;
|
||||
}
|
||||
out.put(
|
||||
e.getKey(),
|
||||
e.getValue() instanceof Map<?, ?> m && depth < MAX_DEPTH
|
||||
@@ -100,6 +111,14 @@ public class SecretMasker {
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if (isSensitiveContainer(key)
|
||||
&& depth < MAX_DEPTH
|
||||
&& stored.get(key) instanceof Map<?, ?> s
|
||||
&& value instanceof Map<?, ?> i) {
|
||||
// Every value here is a secret, so restore a redacted one from stored per-entry.
|
||||
out.put(key, mergeAllValues(castMap(s), castMap(i), depth + 1));
|
||||
continue;
|
||||
}
|
||||
if (depth < MAX_DEPTH
|
||||
&& stored.get(key) instanceof Map<?, ?> s
|
||||
&& value instanceof Map<?, ?> i) {
|
||||
@@ -119,6 +138,9 @@ public class SecretMasker {
|
||||
}
|
||||
return MASK;
|
||||
}
|
||||
if (isSensitiveContainer(key) && value instanceof Map<?, ?> m && depth < MAX_DEPTH) {
|
||||
return maskAllValues(castMap(m), depth + 1);
|
||||
}
|
||||
if (depth >= MAX_DEPTH) {
|
||||
// Too deep to descend; mask containers rather than risk leaking an unmasked secret.
|
||||
return value instanceof Map<?, ?> || value instanceof List<?> ? MASK : value;
|
||||
@@ -141,6 +163,53 @@ public class SecretMasker {
|
||||
return SENSITIVE_HINTS.stream().anyMatch(lower::contains);
|
||||
}
|
||||
|
||||
private boolean isSensitiveContainer(String key) {
|
||||
return SENSITIVE_VALUE_CONTAINERS.contains(key.toLowerCase(Locale.ROOT));
|
||||
}
|
||||
|
||||
/** Mask every value in a container map, whatever its keys are named. */
|
||||
private Map<String, Object> maskAllValues(Map<String, Object> map, int depth) {
|
||||
Map<String, Object> out = new LinkedHashMap<>();
|
||||
for (Map.Entry<String, Object> e : map.entrySet()) {
|
||||
Object v = e.getValue();
|
||||
if (v == null || (v instanceof String s && s.isBlank())) {
|
||||
out.put(e.getKey(), v);
|
||||
} else if (v instanceof Map<?, ?> m && depth < MAX_DEPTH) {
|
||||
out.put(e.getKey(), maskAllValues(castMap(m), depth + 1));
|
||||
} else {
|
||||
out.put(e.getKey(), MASK);
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/** Merge a container map treating every entry as a secret, restoring redacted from stored. */
|
||||
private Map<String, Object> mergeAllValues(
|
||||
Map<String, Object> stored, Map<String, Object> incoming, int depth) {
|
||||
Map<String, Object> out = new LinkedHashMap<>();
|
||||
for (Map.Entry<String, Object> e : incoming.entrySet()) {
|
||||
if (isRedacted(e.getValue(), depth)) {
|
||||
if (stored.containsKey(e.getKey())) {
|
||||
out.put(e.getKey(), stored.get(e.getKey()));
|
||||
}
|
||||
} else {
|
||||
out.put(e.getKey(), e.getValue());
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/** Drop redacted entries from a container map on create, whatever their keys are named. */
|
||||
private Map<String, Object> sanitizeAllValues(Map<String, Object> map, int depth) {
|
||||
Map<String, Object> out = new LinkedHashMap<>();
|
||||
for (Map.Entry<String, Object> e : map.entrySet()) {
|
||||
if (!isRedacted(e.getValue(), depth)) {
|
||||
out.put(e.getKey(), e.getValue());
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/** Blank, the mask placeholder, or any structure that still contains the mask. */
|
||||
private boolean isRedacted(Object value, int depth) {
|
||||
if (value == null) {
|
||||
|
||||
+8
@@ -0,0 +1,8 @@
|
||||
package stirling.software.proprietary.classification;
|
||||
|
||||
/** Meters a client-side classification run; SaaS charges PAYG, other flavors have no bean. */
|
||||
public interface ClassificationRunBiller {
|
||||
|
||||
/** Charge one classification policy run covering {@code documentCount} documents. */
|
||||
void recordClassificationRun(int documentCount);
|
||||
}
|
||||
+13
-2
@@ -54,10 +54,21 @@ public class CustomAuditEventRepository implements AuditEventRepository {
|
||||
return;
|
||||
}
|
||||
String rid = MDC.get("requestId");
|
||||
String apiKeyLabel =
|
||||
MDC.get(
|
||||
stirling.software.proprietary.security.service
|
||||
.ApiKeyAuthenticationService.AUDIT_LABEL_MDC_KEY);
|
||||
|
||||
if (rid != null) {
|
||||
if (rid != null || apiKeyLabel != null) {
|
||||
clean = new java.util.HashMap<>(clean);
|
||||
clean.put("requestId", rid);
|
||||
if (rid != null) {
|
||||
clean.put("requestId", rid);
|
||||
}
|
||||
// Named key that made the request; surfaces as the doc source in the processor
|
||||
// feed.
|
||||
if (apiKeyLabel != null) {
|
||||
clean.put("__apiKeyLabel", apiKeyLabel);
|
||||
}
|
||||
}
|
||||
|
||||
String source = MDC.get("auditSource");
|
||||
|
||||
+15
-6
@@ -7,7 +7,6 @@ import java.util.concurrent.Executor;
|
||||
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import org.springframework.beans.factory.annotation.Qualifier;
|
||||
import org.springframework.beans.factory.annotation.Value;
|
||||
import org.springframework.http.HttpStatus;
|
||||
import org.springframework.http.MediaType;
|
||||
import org.springframework.http.ResponseEntity;
|
||||
@@ -28,6 +27,7 @@ import jakarta.validation.Valid;
|
||||
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
import stirling.software.common.model.ApplicationProperties;
|
||||
import stirling.software.common.model.job.ResultFile;
|
||||
import stirling.software.common.service.JobOwnershipService;
|
||||
import stirling.software.common.service.TaskManager;
|
||||
@@ -38,6 +38,7 @@ import stirling.software.proprietary.model.api.ai.AiWorkflowResponse;
|
||||
import stirling.software.proprietary.model.api.ai.AiWorkflowResultFile;
|
||||
import stirling.software.proprietary.service.AiEngineClient;
|
||||
import stirling.software.proprietary.service.AiEngineEndpointResolver;
|
||||
import stirling.software.proprietary.service.AiFeatureGate;
|
||||
import stirling.software.proprietary.service.AiWorkflowService;
|
||||
|
||||
import tools.jackson.core.JacksonException;
|
||||
@@ -60,15 +61,14 @@ public class AiEngineController {
|
||||
private final TaskManager taskManager;
|
||||
private final JobOwnershipService jobOwnershipService;
|
||||
private final AiEngineEndpointResolver endpointResolver;
|
||||
private final AiFeatureGate aiFeatureGate;
|
||||
private final UserServiceInterface userService;
|
||||
|
||||
/**
|
||||
* SSE emitter timeout. Long enough to accommodate multi-gigabyte PDF workflows (OCR on a
|
||||
* 1000-page scan, splitting a huge PDF, etc.) without the emitter completing out from under the
|
||||
* executor. Configurable via {@code stirling.ai.streamTimeoutMs}.
|
||||
* SSE emitter timeout (ms), long enough for multi-gigabyte PDF workflows without completing out
|
||||
* from under the executor. Derived from {@code aiEngine.streamTimeoutSeconds}.
|
||||
*/
|
||||
@Value("${stirling.ai.streamTimeoutMs:1800000}")
|
||||
private long streamTimeoutMs;
|
||||
private final long streamTimeoutMs;
|
||||
|
||||
public AiEngineController(
|
||||
AiEngineClient aiEngineClient,
|
||||
@@ -78,6 +78,8 @@ public class AiEngineController {
|
||||
TaskManager taskManager,
|
||||
JobOwnershipService jobOwnershipService,
|
||||
AiEngineEndpointResolver endpointResolver,
|
||||
AiFeatureGate aiFeatureGate,
|
||||
ApplicationProperties applicationProperties,
|
||||
@Autowired(required = false) UserServiceInterface userService) {
|
||||
this.aiEngineClient = aiEngineClient;
|
||||
this.aiWorkflowService = aiWorkflowService;
|
||||
@@ -86,7 +88,10 @@ public class AiEngineController {
|
||||
this.taskManager = taskManager;
|
||||
this.jobOwnershipService = jobOwnershipService;
|
||||
this.endpointResolver = endpointResolver;
|
||||
this.aiFeatureGate = aiFeatureGate;
|
||||
this.userService = userService;
|
||||
this.streamTimeoutMs =
|
||||
applicationProperties.getAiEngine().getStreamTimeoutSeconds() * 1000L;
|
||||
}
|
||||
|
||||
private String currentUserId() {
|
||||
@@ -111,6 +116,7 @@ public class AiEngineController {
|
||||
+ " system and downloadable via GET /api/v1/general/files/{fileId}.")
|
||||
public AiWorkflowResponse orchestrate(@Valid @ModelAttribute AiWorkflowRequest request)
|
||||
throws IOException {
|
||||
aiFeatureGate.requireConversationalWorkflow();
|
||||
AiWorkflowResponse result = aiWorkflowService.orchestrate(request);
|
||||
registerFileResultAsJob(result);
|
||||
return result;
|
||||
@@ -123,6 +129,7 @@ public class AiEngineController {
|
||||
"Accepts a PDF upload and a user message, returns SSE events with progress"
|
||||
+ " updates followed by the final AI workflow result")
|
||||
public SseEmitter orchestrateStream(@Valid @ModelAttribute AiWorkflowRequest request) {
|
||||
aiFeatureGate.requireConversationalWorkflow();
|
||||
SseEmitter emitter = new SseEmitter(streamTimeoutMs);
|
||||
|
||||
emitter.onTimeout(
|
||||
@@ -246,6 +253,8 @@ public class AiEngineController {
|
||||
"Sends a user message to the PDF edit agent which returns a structured plan"
|
||||
+ " of tool operations to perform")
|
||||
public ResponseEntity<String> pdfEdit(@RequestBody String requestBody) throws IOException {
|
||||
// Same gate as /orchestrate: edit agent is a model call on the same conversational surface.
|
||||
aiFeatureGate.requireConversationalWorkflow();
|
||||
JsonNode parsed = parseJson(requestBody);
|
||||
if (!parsed.isObject()) {
|
||||
throw new ResponseStatusException(
|
||||
|
||||
+5
@@ -35,6 +35,7 @@ import stirling.software.proprietary.classification.ClassificationLabelProvider;
|
||||
import stirling.software.proprietary.classification.model.ClassificationLabel;
|
||||
import stirling.software.proprietary.model.api.ai.AiPageText;
|
||||
import stirling.software.proprietary.service.AiEngineClient;
|
||||
import stirling.software.proprietary.service.AiFeatureGate;
|
||||
import stirling.software.proprietary.service.PdfContentExtractor;
|
||||
|
||||
import tools.jackson.databind.JsonNode;
|
||||
@@ -67,6 +68,7 @@ public class ClassifyLabelController {
|
||||
private final PdfContentExtractor pdfContentExtractor;
|
||||
private final PdfMetadataService pdfMetadataService;
|
||||
private final AiEngineClient aiEngineClient;
|
||||
private final AiFeatureGate aiFeatureGate;
|
||||
private final ObjectMapper objectMapper;
|
||||
private final UserServiceInterface userService;
|
||||
|
||||
@@ -81,6 +83,7 @@ public class ClassifyLabelController {
|
||||
PdfContentExtractor pdfContentExtractor,
|
||||
PdfMetadataService pdfMetadataService,
|
||||
AiEngineClient aiEngineClient,
|
||||
AiFeatureGate aiFeatureGate,
|
||||
ObjectMapper objectMapper,
|
||||
ClassificationLabelProvider labelProvider,
|
||||
@Autowired(required = false) UserServiceInterface userService) {
|
||||
@@ -89,6 +92,7 @@ public class ClassifyLabelController {
|
||||
this.pdfContentExtractor = pdfContentExtractor;
|
||||
this.pdfMetadataService = pdfMetadataService;
|
||||
this.aiEngineClient = aiEngineClient;
|
||||
this.aiFeatureGate = aiFeatureGate;
|
||||
this.objectMapper = objectMapper;
|
||||
this.labelProvider = labelProvider;
|
||||
this.userService = userService;
|
||||
@@ -104,6 +108,7 @@ public class ClassifyLabelController {
|
||||
+ " intended for direct client use.")
|
||||
public ResponseEntity<Resource> classifyAndLabel(
|
||||
@RequestParam("fileInput") MultipartFile fileInput) throws IOException {
|
||||
aiFeatureGate.requireClassify();
|
||||
try (PDDocument document = pdfDocumentFactory.load(fileInput, true)) {
|
||||
String fileName = safeFileName(fileInput.getOriginalFilename());
|
||||
|
||||
|
||||
+3
-1
@@ -34,6 +34,7 @@ import stirling.software.common.util.TempFileManager;
|
||||
import stirling.software.common.util.WebResponseUtils;
|
||||
import stirling.software.proprietary.model.api.ai.create.AiDocument;
|
||||
import stirling.software.proprietary.service.AiDocumentHtmlRenderer;
|
||||
import stirling.software.proprietary.service.AiFeatureGate;
|
||||
|
||||
import tools.jackson.core.JacksonException;
|
||||
import tools.jackson.databind.ObjectMapper;
|
||||
@@ -59,6 +60,7 @@ public class CreatePdfAgentController {
|
||||
private final ApplicationProperties applicationProperties;
|
||||
private final ObjectMapper objectMapper;
|
||||
private final AiDocumentHtmlRenderer htmlRenderer;
|
||||
private final AiFeatureGate aiFeatureGate;
|
||||
|
||||
/**
|
||||
* Returns true only when WeasyPrint is definitively unavailable — either the binary could not
|
||||
@@ -93,10 +95,10 @@ public class CreatePdfAgentController {
|
||||
public ResponseEntity<Resource> createPdf(
|
||||
@RequestParam("document") String document, @RequestParam("filename") String filename)
|
||||
throws Exception {
|
||||
|
||||
if (!applicationProperties.getAiEngine().isEnabled()) {
|
||||
throw new ResponseStatusException(HttpStatus.NOT_FOUND);
|
||||
}
|
||||
aiFeatureGate.requireCreatePdf();
|
||||
|
||||
AiDocument model;
|
||||
try {
|
||||
|
||||
+303
@@ -0,0 +1,303 @@
|
||||
package stirling.software.proprietary.controller.api;
|
||||
|
||||
import java.io.ByteArrayOutputStream;
|
||||
import java.io.IOException;
|
||||
import java.io.StringWriter;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.util.List;
|
||||
import java.util.zip.ZipEntry;
|
||||
import java.util.zip.ZipOutputStream;
|
||||
|
||||
import org.apache.commons.csv.CSVFormat;
|
||||
import org.apache.commons.csv.CSVPrinter;
|
||||
import org.springframework.core.io.ByteArrayResource;
|
||||
import org.springframework.core.io.Resource;
|
||||
import org.springframework.http.HttpHeaders;
|
||||
import org.springframework.http.MediaType;
|
||||
import org.springframework.http.ResponseEntity;
|
||||
import org.springframework.web.bind.annotation.GetMapping;
|
||||
import org.springframework.web.bind.annotation.ModelAttribute;
|
||||
import org.springframework.web.bind.annotation.RequestMapping;
|
||||
import org.springframework.web.bind.annotation.RestController;
|
||||
import org.springframework.web.multipart.MultipartFile;
|
||||
|
||||
import io.swagger.v3.oas.annotations.Operation;
|
||||
import io.swagger.v3.oas.annotations.tags.Tag;
|
||||
|
||||
import lombok.RequiredArgsConstructor;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
import stirling.software.common.annotations.AutoJobPostMapping;
|
||||
import stirling.software.common.enumeration.ResourceWeight;
|
||||
import stirling.software.common.util.GeneralUtils;
|
||||
import stirling.software.common.util.WebResponseUtils;
|
||||
import stirling.software.proprietary.model.api.docparse.ExtractFieldsApiRequest;
|
||||
import stirling.software.proprietary.model.api.docparse.ExtractTablesApiRequest;
|
||||
import stirling.software.proprietary.model.api.docparse.RagIngestApiRequest;
|
||||
import stirling.software.proprietary.model.api.docparse.SuggestSchemaApiRequest;
|
||||
import stirling.software.proprietary.model.docparse.DocChunk;
|
||||
import stirling.software.proprietary.model.docparse.DocTable;
|
||||
import stirling.software.proprietary.model.docparse.DocparseCapabilitiesView;
|
||||
import stirling.software.proprietary.model.docparse.DocparseMode;
|
||||
import stirling.software.proprietary.model.docparse.ExtractFieldsResponse;
|
||||
import stirling.software.proprietary.model.docparse.ExtractTablesResponse;
|
||||
import stirling.software.proprietary.model.docparse.RagIngestResponse;
|
||||
import stirling.software.proprietary.model.docparse.SuggestSchemaResponse;
|
||||
import stirling.software.proprietary.service.AiToolResponseHeaders;
|
||||
import stirling.software.proprietary.service.DocParseService;
|
||||
|
||||
import tools.jackson.databind.ObjectMapper;
|
||||
import tools.jackson.databind.node.ObjectNode;
|
||||
|
||||
/**
|
||||
* Public DocParse ingestion API. Thin HTTP layer over {@link DocParseService}, which owns the
|
||||
* engine wire contract; this class owns the pipeline step shape (report header, export ZIP).
|
||||
*/
|
||||
@Slf4j
|
||||
@RestController
|
||||
@RequestMapping("/api/v1/docparse")
|
||||
@RequiredArgsConstructor
|
||||
@Tag(
|
||||
name = "DocParse",
|
||||
description =
|
||||
"Document ingestion: chunk, embed, and index documents into the searchable"
|
||||
+ " knowledge base, or export the parsed content (markdown, chunks JSONL)"
|
||||
+ " for external systems.")
|
||||
public class DocParseController {
|
||||
|
||||
private static final MediaType CSV = MediaType.parseMediaType("text/csv");
|
||||
|
||||
private final DocParseService docParseService;
|
||||
private final ObjectMapper objectMapper;
|
||||
|
||||
@AutoJobPostMapping(
|
||||
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
|
||||
value = "/rag-ingest",
|
||||
resourceWeight = ResourceWeight.LARGE_WEIGHT)
|
||||
@Operation(
|
||||
summary = "Chunk, embed, and index a document into the RAG store (pipeline shape)",
|
||||
description =
|
||||
"Ingests the document into the engine's RAG store under a stable documentId"
|
||||
+ " (default: content hash). Returns the ORIGINAL PDF unchanged as the"
|
||||
+ " body, with the ingest summary JSON in the X-Stirling-Tool-Report"
|
||||
+ " header so policy pipelines pick it up as the step report. With"
|
||||
+ " exportMarkdown/exportChunksJsonl the body becomes a ZIP holding the"
|
||||
+ " original plus the corpus files, ready for delivery to external"
|
||||
+ " systems. Input:PDF Output:PDF/ZIP Type:SISO")
|
||||
public ResponseEntity<Resource> ragIngest(@ModelAttribute RagIngestApiRequest request)
|
||||
throws IOException {
|
||||
MultipartFile file = request.getFileInput();
|
||||
boolean export = request.isExportMarkdown() || request.isExportChunksJsonl();
|
||||
RagIngestResponse result =
|
||||
docParseService.ragIngest(
|
||||
file,
|
||||
request.getDocumentId(),
|
||||
request.getChunkSize(),
|
||||
request.getOverlap(),
|
||||
DocparseMode.fromWire(request.getMode()),
|
||||
request.isIndex(),
|
||||
request.isExportMarkdown(),
|
||||
request.isExportChunksJsonl());
|
||||
|
||||
// The report header must stay small: summary fields only, never the echoed content.
|
||||
ObjectNode report = objectMapper.createObjectNode();
|
||||
report.put("mode", result.mode().wire());
|
||||
report.put("documentId", result.documentId());
|
||||
report.put("chunksIndexed", result.chunksIndexed());
|
||||
report.put("pages", result.pages());
|
||||
report.put("indexed", request.isIndex());
|
||||
|
||||
String fileName = DocParseService.fileName(file);
|
||||
byte[] original = file.getBytes();
|
||||
HttpHeaders headers = new HttpHeaders();
|
||||
headers.set(AiToolResponseHeaders.TOOL_REPORT, objectMapper.writeValueAsString(report));
|
||||
|
||||
if (!export) {
|
||||
headers.setContentType(MediaType.APPLICATION_PDF);
|
||||
headers.setContentDispositionFormData("attachment", fileName);
|
||||
headers.setContentLength(original.length);
|
||||
return ResponseEntity.ok().headers(headers).body(new ByteArrayResource(original));
|
||||
}
|
||||
|
||||
byte[] zip = exportZip(fileName, original, result, request);
|
||||
headers.setContentType(MediaType.parseMediaType("application/zip"));
|
||||
headers.setContentDispositionFormData("attachment", baseName(fileName) + "-ingested.zip");
|
||||
headers.setContentLength(zip.length);
|
||||
return ResponseEntity.ok().headers(headers).body(new ByteArrayResource(zip));
|
||||
}
|
||||
|
||||
@AutoJobPostMapping(
|
||||
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
|
||||
value = "/extract-fields",
|
||||
resourceWeight = ResourceWeight.LARGE_WEIGHT)
|
||||
@Operation(
|
||||
summary = "Extract typed fields from a document (pipeline shape)",
|
||||
description =
|
||||
"Extracts the fields described by the JSON Schema and returns the ORIGINAL PDF"
|
||||
+ " unchanged as the body, with the extraction JSON in the"
|
||||
+ " X-Stirling-Tool-Report header so policy pipelines pick it up as the"
|
||||
+ " step report. Use /extract-fields/json for the raw JSON."
|
||||
+ " Input:PDF Output:PDF Type:SISO")
|
||||
public ResponseEntity<Resource> extractFields(@ModelAttribute ExtractFieldsApiRequest request)
|
||||
throws IOException {
|
||||
MultipartFile file = request.getFileInput();
|
||||
ExtractFieldsResponse result =
|
||||
docParseService.extractFields(
|
||||
file,
|
||||
request.getFieldsSchema(),
|
||||
DocparseMode.fromWire(request.getMode()),
|
||||
request.getInstructions());
|
||||
byte[] original = file.getBytes();
|
||||
HttpHeaders headers = new HttpHeaders();
|
||||
headers.setContentType(MediaType.APPLICATION_PDF);
|
||||
headers.setContentDispositionFormData("attachment", DocParseService.fileName(file));
|
||||
headers.setContentLength(original.length);
|
||||
headers.set(AiToolResponseHeaders.TOOL_REPORT, objectMapper.writeValueAsString(result));
|
||||
return ResponseEntity.ok().headers(headers).body(new ByteArrayResource(original));
|
||||
}
|
||||
|
||||
@AutoJobPostMapping(
|
||||
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
|
||||
value = "/extract-fields/json",
|
||||
resourceWeight = ResourceWeight.LARGE_WEIGHT)
|
||||
@Operation(
|
||||
summary = "Extract typed fields from a document (JSON)",
|
||||
description =
|
||||
"Extracts the fields described by the JSON Schema and returns the extraction"
|
||||
+ " result (fields, confidence, citations) as JSON."
|
||||
+ " Input:PDF Output:JSON Type:SISO")
|
||||
public ResponseEntity<ExtractFieldsResponse> extractFieldsJson(
|
||||
@ModelAttribute ExtractFieldsApiRequest request) throws IOException {
|
||||
return ResponseEntity.ok(
|
||||
docParseService.extractFields(
|
||||
request.getFileInput(),
|
||||
request.getFieldsSchema(),
|
||||
DocparseMode.fromWire(request.getMode()),
|
||||
request.getInstructions()));
|
||||
}
|
||||
|
||||
@AutoJobPostMapping(
|
||||
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
|
||||
value = "/suggest-schema",
|
||||
resourceWeight = ResourceWeight.LARGE_WEIGHT)
|
||||
@Operation(
|
||||
summary = "Suggest an extraction schema for a document",
|
||||
description =
|
||||
"Reads the document and proposes the fields worth extracting (name, type,"
|
||||
+ " description), ready to feed into /extract-fields as a JSON Schema."
|
||||
+ " Input:PDF Output:JSON Type:SISO")
|
||||
public ResponseEntity<SuggestSchemaResponse> suggestSchema(
|
||||
@ModelAttribute SuggestSchemaApiRequest request) throws IOException {
|
||||
return ResponseEntity.ok(
|
||||
docParseService.suggestSchema(request.getFileInput(), request.getMaxFields()));
|
||||
}
|
||||
|
||||
@GetMapping("/capabilities")
|
||||
@Operation(
|
||||
summary = "DocParse capability summary",
|
||||
description =
|
||||
"Merged view of the Java settings and the engine's capability probe, so"
|
||||
+ " clients can gate advanced-tier UI.")
|
||||
public ResponseEntity<DocparseCapabilitiesView> capabilities() {
|
||||
return ResponseEntity.ok(docParseService.capabilitiesView());
|
||||
}
|
||||
|
||||
@AutoJobPostMapping(
|
||||
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
|
||||
value = "/extract-tables",
|
||||
resourceWeight = ResourceWeight.LARGE_WEIGHT)
|
||||
@Operation(
|
||||
summary = "Extract tables from a document",
|
||||
description =
|
||||
"Extracts table structure and returns CSV (all tables concatenated, blank line"
|
||||
+ " between them) or the structured JSON table list."
|
||||
+ " Input:PDF Output:CSV/JSON Type:SISO")
|
||||
public ResponseEntity<?> extractTables(@ModelAttribute ExtractTablesApiRequest request)
|
||||
throws IOException {
|
||||
ExtractTablesResponse result = docParseService.tables(request.getFileInput());
|
||||
if ("json".equalsIgnoreCase(request.getOutputFormat())) {
|
||||
return ResponseEntity.ok(result);
|
||||
}
|
||||
return WebResponseUtils.bytesToWebResponse(
|
||||
tablesToCsv(result.tables()).getBytes(StandardCharsets.UTF_8),
|
||||
outputName(request.getFileInput(), "_tables.csv"),
|
||||
CSV);
|
||||
}
|
||||
|
||||
/** Original + requested corpus files in one ZIP, so destinations receive them together. */
|
||||
private byte[] exportZip(
|
||||
String fileName, byte[] original, RagIngestResponse result, RagIngestApiRequest request)
|
||||
throws IOException {
|
||||
String base = baseName(fileName);
|
||||
ByteArrayOutputStream out = new ByteArrayOutputStream();
|
||||
try (ZipOutputStream zip = new ZipOutputStream(out)) {
|
||||
zip.putNextEntry(new ZipEntry(fileName));
|
||||
zip.write(original);
|
||||
zip.closeEntry();
|
||||
if (request.isExportMarkdown()) {
|
||||
zip.putNextEntry(new ZipEntry(base + ".md"));
|
||||
zip.write(
|
||||
(result.markdown() == null ? "" : result.markdown())
|
||||
.getBytes(StandardCharsets.UTF_8));
|
||||
zip.closeEntry();
|
||||
}
|
||||
if (request.isExportChunksJsonl()) {
|
||||
zip.putNextEntry(new ZipEntry(base + ".chunks.jsonl"));
|
||||
zip.write(chunksJsonl(result).getBytes(StandardCharsets.UTF_8));
|
||||
zip.closeEntry();
|
||||
}
|
||||
}
|
||||
return out.toByteArray();
|
||||
}
|
||||
|
||||
/** One chunk per line, each self-describing (documentId + source travel on every line). */
|
||||
private String chunksJsonl(RagIngestResponse result) {
|
||||
if (result.chunks() == null) {
|
||||
return "";
|
||||
}
|
||||
StringBuilder lines = new StringBuilder();
|
||||
for (DocChunk chunk : result.chunks()) {
|
||||
ObjectNode line = objectMapper.createObjectNode();
|
||||
line.put("documentId", result.documentId());
|
||||
line.put("index", chunk.index());
|
||||
line.put("text", chunk.text());
|
||||
if (chunk.pageStart() != null) {
|
||||
line.put("pageStart", chunk.pageStart());
|
||||
}
|
||||
if (chunk.pageEnd() != null) {
|
||||
line.put("pageEnd", chunk.pageEnd());
|
||||
}
|
||||
var headings = line.putArray("headingPath");
|
||||
chunk.headingPath().forEach(headings::add);
|
||||
lines.append(objectMapper.writeValueAsString(line)).append('\n');
|
||||
}
|
||||
return lines.toString();
|
||||
}
|
||||
|
||||
private static String baseName(String fileName) {
|
||||
int dot = fileName.lastIndexOf('.');
|
||||
return dot > 0 ? fileName.substring(0, dot) : fileName;
|
||||
}
|
||||
|
||||
private static String tablesToCsv(List<DocTable> tables) throws IOException {
|
||||
CSVFormat format = CSVFormat.EXCEL.builder().setEscape('"').build();
|
||||
StringWriter writer = new StringWriter();
|
||||
try (CSVPrinter printer = format.print(writer)) {
|
||||
boolean first = true;
|
||||
for (DocTable table : tables) {
|
||||
if (!first) {
|
||||
printer.println();
|
||||
}
|
||||
first = false;
|
||||
for (List<String> row : table.cells()) {
|
||||
printer.printRecord(row);
|
||||
}
|
||||
}
|
||||
}
|
||||
return writer.toString();
|
||||
}
|
||||
|
||||
private static String outputName(MultipartFile file, String suffix) {
|
||||
return GeneralUtils.removeExtension(DocParseService.fileName(file)) + suffix;
|
||||
}
|
||||
}
|
||||
+8
-2
@@ -2,6 +2,7 @@ package stirling.software.proprietary.controller.api;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.math.BigDecimal;
|
||||
import java.util.regex.Pattern;
|
||||
|
||||
import org.springframework.http.HttpStatus;
|
||||
import org.springframework.http.MediaType;
|
||||
@@ -20,6 +21,7 @@ import lombok.RequiredArgsConstructor;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
import stirling.software.proprietary.model.api.ai.Verdict;
|
||||
import stirling.software.proprietary.service.AiFeatureGate;
|
||||
import stirling.software.proprietary.service.AiToolInputValidator;
|
||||
import stirling.software.proprietary.service.MathAuditorOrchestrator;
|
||||
|
||||
@@ -46,7 +48,9 @@ import stirling.software.proprietary.service.MathAuditorOrchestrator;
|
||||
@Tag(name = "AI Tools", description = "Dispatchable AI-backed tools.")
|
||||
public class MathAuditorAgentController {
|
||||
|
||||
private static final Pattern NEWLINE_PATTERN = Pattern.compile("[\\r\\n]");
|
||||
private final MathAuditorOrchestrator orchestrator;
|
||||
private final AiFeatureGate aiFeatureGate;
|
||||
|
||||
@PostMapping(value = "/math-auditor-agent", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
|
||||
@Operation(
|
||||
@@ -77,15 +81,17 @@ public class MathAuditorAgentController {
|
||||
+ " ignored (default: 0.01)")
|
||||
@RequestParam(value = "tolerance", defaultValue = "0.01")
|
||||
BigDecimal tolerance) {
|
||||
aiFeatureGate.requireMathAuditor();
|
||||
|
||||
AiToolInputValidator.validatePdfUpload(fileInput);
|
||||
if (tolerance.compareTo(BigDecimal.ZERO) < 0) {
|
||||
return ResponseEntity.badRequest().build();
|
||||
}
|
||||
|
||||
String originalFilename = fileInput.getOriginalFilename();
|
||||
String safeName =
|
||||
fileInput.getOriginalFilename() != null
|
||||
? fileInput.getOriginalFilename().replaceAll("[\\r\\n]", "_")
|
||||
originalFilename != null
|
||||
? NEWLINE_PATTERN.matcher(originalFilename).replaceAll("_")
|
||||
: "<unnamed>";
|
||||
log.info("[math-auditor-agent] request file={} tolerance={}", safeName, tolerance);
|
||||
|
||||
|
||||
+8
-2
@@ -1,6 +1,7 @@
|
||||
package stirling.software.proprietary.controller.api;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.util.regex.Pattern;
|
||||
|
||||
import org.springframework.core.io.ByteArrayResource;
|
||||
import org.springframework.core.io.Resource;
|
||||
@@ -20,6 +21,7 @@ import io.swagger.v3.oas.annotations.tags.Tag;
|
||||
import lombok.RequiredArgsConstructor;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
import stirling.software.proprietary.service.AiFeatureGate;
|
||||
import stirling.software.proprietary.service.AiToolResponseHeaders;
|
||||
import stirling.software.proprietary.service.PdfCommentAgentOrchestrator;
|
||||
import stirling.software.proprietary.service.PdfCommentAgentOrchestrator.AnnotatedPdf;
|
||||
@@ -45,8 +47,10 @@ import tools.jackson.databind.node.ObjectNode;
|
||||
@Tag(name = "AI Tools", description = "Dispatchable AI-backed tools.")
|
||||
public class PdfCommentAgentController {
|
||||
|
||||
private static final Pattern NEWLINE_PATTERN = Pattern.compile("[\\r\\n]");
|
||||
private final PdfCommentAgentOrchestrator orchestrator;
|
||||
private final ObjectMapper objectMapper;
|
||||
private final AiFeatureGate aiFeatureGate;
|
||||
|
||||
@PostMapping(
|
||||
value = "/pdf-comment-agent",
|
||||
@@ -77,10 +81,12 @@ public class PdfCommentAgentController {
|
||||
@RequestParam("prompt")
|
||||
String prompt)
|
||||
throws IOException {
|
||||
aiFeatureGate.requirePdfComment();
|
||||
|
||||
String originalFilename = fileInput.getOriginalFilename();
|
||||
String safeName =
|
||||
fileInput.getOriginalFilename() != null
|
||||
? fileInput.getOriginalFilename().replaceAll("[\\r\\n]", "_")
|
||||
originalFilename != null
|
||||
? NEWLINE_PATTERN.matcher(originalFilename).replaceAll("_")
|
||||
: "<unnamed>";
|
||||
log.info(
|
||||
"[pdf-comment-agent] request file={} promptLen={}",
|
||||
|
||||
+54
@@ -0,0 +1,54 @@
|
||||
package stirling.software.proprietary.controller.api;
|
||||
|
||||
import org.springframework.http.ResponseEntity;
|
||||
import org.springframework.web.bind.annotation.DeleteMapping;
|
||||
import org.springframework.web.bind.annotation.GetMapping;
|
||||
import org.springframework.web.bind.annotation.PathVariable;
|
||||
import org.springframework.web.bind.annotation.PostMapping;
|
||||
import org.springframework.web.bind.annotation.RequestBody;
|
||||
import org.springframework.web.bind.annotation.RequestParam;
|
||||
|
||||
import io.swagger.v3.oas.annotations.Operation;
|
||||
|
||||
import lombok.RequiredArgsConstructor;
|
||||
|
||||
import stirling.software.common.annotations.api.ProprietaryUiDataApi;
|
||||
import stirling.software.proprietary.model.api.apikey.CreateApiKeyRequest;
|
||||
import stirling.software.proprietary.model.api.apikey.CreatedApiKeyDto;
|
||||
import stirling.software.proprietary.model.api.apikey.PortalApiKeysResponse;
|
||||
import stirling.software.proprietary.security.service.ApiKeyManagementService;
|
||||
|
||||
/**
|
||||
* Real backing for the portal Infrastructure → API Keys tab: list/create/revoke named, personal API
|
||||
* keys. Replaces the former portal-only mock endpoint. Not gated behind an Enterprise license - API
|
||||
* keys are a core auth feature available on every self-hosted instance.
|
||||
*/
|
||||
@ProprietaryUiDataApi
|
||||
@RequiredArgsConstructor
|
||||
public class PortalApiKeysController {
|
||||
|
||||
private final ApiKeyManagementService apiKeyManagementService;
|
||||
|
||||
// tier accepted for endpoint symmetry with the other infra tabs; ignored here.
|
||||
@GetMapping("/infrastructure/api-keys")
|
||||
@Operation(summary = "List API keys", description = "The caller's personal API keys.")
|
||||
public ResponseEntity<PortalApiKeysResponse> list(
|
||||
@RequestParam(value = "tier", required = false) String tier) {
|
||||
return ResponseEntity.ok(apiKeyManagementService.listVisibleKeys());
|
||||
}
|
||||
|
||||
@PostMapping("/infrastructure/api-keys")
|
||||
@Operation(
|
||||
summary = "Create an API key",
|
||||
description = "Mints a personal key and returns its one-time secret.")
|
||||
public ResponseEntity<CreatedApiKeyDto> create(@RequestBody CreateApiKeyRequest request) {
|
||||
return ResponseEntity.ok(apiKeyManagementService.createKey(request));
|
||||
}
|
||||
|
||||
@DeleteMapping("/infrastructure/api-keys/{id}")
|
||||
@Operation(summary = "Revoke an API key", description = "Disables a key the caller owns.")
|
||||
public ResponseEntity<Void> revoke(@PathVariable("id") Long id) {
|
||||
apiKeyManagementService.revokeKey(id);
|
||||
return ResponseEntity.noContent().build();
|
||||
}
|
||||
}
|
||||
+112
-103
@@ -3,7 +3,6 @@ package stirling.software.proprietary.controller.api;
|
||||
import static stirling.software.common.util.ProviderUtils.validateProvider;
|
||||
|
||||
import java.time.Instant;
|
||||
import java.time.temporal.ChronoUnit;
|
||||
import java.util.*;
|
||||
import java.util.stream.Collectors;
|
||||
|
||||
@@ -45,7 +44,6 @@ import stirling.software.proprietary.security.config.EnterpriseEndpoint;
|
||||
import stirling.software.proprietary.security.database.repository.SessionRepository;
|
||||
import stirling.software.proprietary.security.database.repository.UserRepository;
|
||||
import stirling.software.proprietary.security.model.Authority;
|
||||
import stirling.software.proprietary.security.model.SessionEntity;
|
||||
import stirling.software.proprietary.security.model.User;
|
||||
import stirling.software.proprietary.security.model.dto.AdminUserSummary;
|
||||
import stirling.software.proprietary.security.repository.TeamMembershipRepository;
|
||||
@@ -169,16 +167,8 @@ public class ProprietaryUIDataController {
|
||||
boolean isFirstTimeSetup = false;
|
||||
boolean showDefaultCredentials = false;
|
||||
|
||||
List<User> allUsers = userRepository.findAll();
|
||||
List<User> realUsers =
|
||||
allUsers.stream()
|
||||
.filter(
|
||||
user ->
|
||||
!Role.INTERNAL_API_USER
|
||||
.getRoleId()
|
||||
.equals(user.getUsername()))
|
||||
.toList();
|
||||
long userCount = realUsers.size();
|
||||
// Count real users, excluding the internal API user.
|
||||
long userCount = userRepository.countByUsernameNot(Role.INTERNAL_API_USER.getRoleId());
|
||||
|
||||
if (userCount == 0) {
|
||||
isFirstTimeSetup = true;
|
||||
@@ -265,92 +255,67 @@ public class ProprietaryUIDataController {
|
||||
@PreAuthorize("hasRole('ADMIN')")
|
||||
@Operation(summary = "Get admin settings data")
|
||||
public ResponseEntity<AdminSettingsData> getAdminSettingsData(Authentication authentication) {
|
||||
List<User> allUsers = userRepository.findAllWithTeam();
|
||||
Iterator<User> iterator = allUsers.iterator();
|
||||
List<User> allUsers = userRepository.findAllWithTeamAndAuthorities();
|
||||
Map<String, String> roleDetails = Role.getAllRoleDetails();
|
||||
|
||||
// Drop the internal API user and internal-team members; the roster never shows them.
|
||||
boolean hasInternalApiUser = false;
|
||||
List<User> visibleUsers = new ArrayList<>(allUsers.size());
|
||||
for (User user : allUsers) {
|
||||
if (user == null) {
|
||||
continue;
|
||||
}
|
||||
if (isInternalApiUser(user)) {
|
||||
hasInternalApiUser = true;
|
||||
continue;
|
||||
}
|
||||
if (user.getTeam() != null
|
||||
&& TeamService.INTERNAL_TEAM_NAME.equals(user.getTeam().getName())) {
|
||||
continue;
|
||||
}
|
||||
visibleUsers.add(user);
|
||||
}
|
||||
if (hasInternalApiUser) {
|
||||
roleDetails.remove(Role.INTERNAL_API_USER.getRoleId());
|
||||
}
|
||||
|
||||
// All users' settings in one query (mfaSecret masked).
|
||||
Map<Long, Map<String, String>> settingsByUserId =
|
||||
loadSettingsByUserId(visibleUsers.stream().map(User::getId).toList());
|
||||
|
||||
// Active = any non-expired session within the inactivity window; expiry is left to
|
||||
// SessionScheduled.
|
||||
int maxInactiveInterval = sessionPersistentRegistry.getMaxInactiveInterval();
|
||||
Instant activeCutoff = Instant.now().minusSeconds(maxInactiveInterval);
|
||||
Map<String, Instant> lastRequestByPrincipal = new HashMap<>();
|
||||
for (Object[] row : sessionRepository.findLatestRequestPerPrincipal()) {
|
||||
if (row[0] != null) {
|
||||
lastRequestByPrincipal.put((String) row[0], (Instant) row[1]);
|
||||
}
|
||||
}
|
||||
Set<String> activePrincipals =
|
||||
new HashSet<>(sessionRepository.findActivePrincipalsSince(activeCutoff));
|
||||
|
||||
Map<String, Boolean> userSessions = new HashMap<>();
|
||||
Map<String, Date> userLastRequest = new HashMap<>();
|
||||
Map<String, Map<String, String>> userSettings = new HashMap<>();
|
||||
int activeUsers = 0;
|
||||
int disabledUsers = 0;
|
||||
|
||||
while (iterator.hasNext()) {
|
||||
User user = iterator.next();
|
||||
if (user != null) {
|
||||
String username = user.getUsername();
|
||||
boolean shouldRemove = false;
|
||||
|
||||
// Check if user is an INTERNAL_API_USER
|
||||
for (Authority authority : user.getAuthorities()) {
|
||||
if (authority.getAuthority().equals(Role.INTERNAL_API_USER.getRoleId())) {
|
||||
shouldRemove = true;
|
||||
roleDetails.remove(Role.INTERNAL_API_USER.getRoleId());
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
// Check if user is part of the Internal team
|
||||
if (user.getTeam() != null
|
||||
&& TeamService.INTERNAL_TEAM_NAME.equals(user.getTeam().getName())) {
|
||||
shouldRemove = true;
|
||||
}
|
||||
|
||||
if (shouldRemove) {
|
||||
iterator.remove();
|
||||
continue;
|
||||
}
|
||||
|
||||
// Session status and last request time
|
||||
int maxInactiveInterval = sessionPersistentRegistry.getMaxInactiveInterval();
|
||||
boolean hasActiveSession = false;
|
||||
Date lastRequest = null;
|
||||
Optional<SessionEntity> latestSession =
|
||||
sessionPersistentRegistry.findLatestSession(username);
|
||||
|
||||
if (latestSession.isPresent()) {
|
||||
SessionEntity sessionEntity = latestSession.get();
|
||||
Instant lastAccessedTime =
|
||||
Optional.ofNullable(sessionEntity.getLastRequest())
|
||||
.orElse(Instant.EPOCH);
|
||||
Instant now = Instant.now();
|
||||
Instant expirationTime =
|
||||
lastAccessedTime.plus(maxInactiveInterval, ChronoUnit.SECONDS);
|
||||
|
||||
if (now.isAfter(expirationTime)) {
|
||||
sessionPersistentRegistry.expireSession(sessionEntity.getSessionId());
|
||||
} else {
|
||||
hasActiveSession = !sessionEntity.isExpired();
|
||||
}
|
||||
lastRequest = Date.from(lastAccessedTime);
|
||||
} else {
|
||||
lastRequest = new Date(0);
|
||||
}
|
||||
|
||||
User userWithSettings =
|
||||
userRepository.findByIdWithSettings(user.getId()).orElse(user);
|
||||
|
||||
// Mask mfaSecret if present in settings
|
||||
Map<String, String> originalSettings = userWithSettings.getSettings();
|
||||
Map<String, String> settingsCopy =
|
||||
originalSettings != null
|
||||
? new HashMap<>(originalSettings)
|
||||
: new HashMap<>();
|
||||
if (settingsCopy.containsKey("mfaSecret")) {
|
||||
settingsCopy.put("mfaSecret", "********");
|
||||
}
|
||||
userSettings.put(username, settingsCopy);
|
||||
userSessions.put(username, hasActiveSession);
|
||||
userLastRequest.put(username, lastRequest);
|
||||
|
||||
if (hasActiveSession) activeUsers++;
|
||||
if (!user.isEnabled()) disabledUsers++;
|
||||
}
|
||||
for (User user : visibleUsers) {
|
||||
String username = user.getUsername();
|
||||
boolean hasActiveSession = activePrincipals.contains(username);
|
||||
Instant lastRequest = lastRequestByPrincipal.get(username);
|
||||
userSessions.put(username, hasActiveSession);
|
||||
userLastRequest.put(
|
||||
username, lastRequest != null ? Date.from(lastRequest) : new Date(0));
|
||||
userSettings.put(username, maskSecrets(settingsByUserId.get(user.getId())));
|
||||
if (hasActiveSession) activeUsers++;
|
||||
if (!user.isEnabled()) disabledUsers++;
|
||||
}
|
||||
|
||||
// Sort users by active status and last request date
|
||||
List<User> sortedUsers =
|
||||
allUsers.stream()
|
||||
visibleUsers.stream()
|
||||
.sorted(
|
||||
(u1, u2) -> {
|
||||
boolean u1Active = userSessions.get(u1.getUsername());
|
||||
@@ -380,11 +345,30 @@ public class ProprietaryUIDataController {
|
||||
int licenseMaxUsers = licenseSettingsService.getSettings().getLicenseMaxUsers();
|
||||
boolean premiumEnabled = applicationProperties.getPremium().isEnabled();
|
||||
|
||||
// Convert User entities to AdminUserSummary DTOs to exclude sensitive fields
|
||||
Set<Long> leaderUserIds = leaderUserIds();
|
||||
// Resolve portal access for the whole roster. The teamLead display flag counts a
|
||||
// LEADER membership on any team (mirrors /me), but the portal default policy only
|
||||
// admits leaders of their own active team, so the bulk check gets the narrower set.
|
||||
List<TeamMembership> leaderMemberships =
|
||||
teamMembershipRepository.findByRoleFetchingUserAndTeam(TeamRole.LEADER);
|
||||
Set<Long> leaderUserIds =
|
||||
leaderMemberships.stream()
|
||||
.map(row -> row.getUser().getId())
|
||||
.collect(Collectors.toSet());
|
||||
Set<Long> activeTeamLeaderUserIds =
|
||||
leaderMemberships.stream()
|
||||
.filter(
|
||||
row ->
|
||||
row.getUser().getTeam() != null
|
||||
&& row.getTeam()
|
||||
.getId()
|
||||
.equals(row.getUser().getTeam().getId()))
|
||||
.map(row -> row.getUser().getId())
|
||||
.collect(Collectors.toSet());
|
||||
Set<Long> portalAccessUserIds =
|
||||
resourceAccessService.usersWithPortalAccess(sortedUsers, activeTeamLeaderUserIds);
|
||||
List<AdminUserSummary> userSummaries =
|
||||
sortedUsers.stream()
|
||||
.map(user -> convertUserToSummary(user, leaderUserIds))
|
||||
.map(user -> convertUserToSummary(user, leaderUserIds, portalAccessUserIds))
|
||||
.toList();
|
||||
|
||||
AdminSettingsData data = new AdminSettingsData();
|
||||
@@ -393,7 +377,7 @@ public class ProprietaryUIDataController {
|
||||
data.setRoleDetails(roleDetails);
|
||||
data.setUserSessions(userSessions);
|
||||
data.setUserLastRequest(userLastRequest);
|
||||
data.setTotalUsers(allUsers.size());
|
||||
data.setTotalUsers(visibleUsers.size());
|
||||
data.setActiveUsers(activeUsers);
|
||||
data.setDisabledUsers(disabledUsers);
|
||||
data.setTeams(allTeams);
|
||||
@@ -516,7 +500,8 @@ public class ProprietaryUIDataController {
|
||||
}
|
||||
|
||||
List<User> teamUsers = userRepository.findAllByTeamId(id);
|
||||
List<User> allUsers = userRepository.findAllWithTeam();
|
||||
// Fetch authorities + team for the available-users list.
|
||||
List<User> allUsers = userRepository.findAllWithTeamAndAuthorities();
|
||||
List<User> availableUsers =
|
||||
allUsers.stream()
|
||||
.filter(
|
||||
@@ -568,24 +553,48 @@ public class ProprietaryUIDataController {
|
||||
return ResponseEntity.ok(data);
|
||||
}
|
||||
|
||||
/** User ids holding a LEADER membership on any team. */
|
||||
private Set<Long> leaderUserIds() {
|
||||
return teamMembershipRepository.findByRoleFetchingUserAndTeam(TeamRole.LEADER).stream()
|
||||
.map(row -> row.getUser().getId())
|
||||
.collect(Collectors.toSet());
|
||||
/** Whether the user holds the internal-API authority (never shown in the roster). */
|
||||
private boolean isInternalApiUser(User user) {
|
||||
for (Authority authority : user.getAuthorities()) {
|
||||
if (Role.INTERNAL_API_USER.getRoleId().equals(authority.getAuthority())) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/** Assemble per-user settings maps from the flat (id, key, value) rows of one bulk query. */
|
||||
private Map<Long, Map<String, String>> loadSettingsByUserId(List<Long> userIds) {
|
||||
Map<Long, Map<String, String>> byUser = new HashMap<>();
|
||||
if (userIds.isEmpty()) {
|
||||
return byUser;
|
||||
}
|
||||
for (Object[] row : userRepository.findSettingsByUserIds(userIds)) {
|
||||
byUser.computeIfAbsent((Long) row[0], id -> new HashMap<>())
|
||||
.put((String) row[1], (String) row[2]);
|
||||
}
|
||||
return byUser;
|
||||
}
|
||||
|
||||
/** Copy a settings map with mfaSecret masked; null-safe. */
|
||||
private Map<String, String> maskSecrets(Map<String, String> settings) {
|
||||
Map<String, String> copy = settings != null ? new HashMap<>(settings) : new HashMap<>();
|
||||
if (copy.containsKey("mfaSecret")) {
|
||||
copy.put("mfaSecret", "********");
|
||||
}
|
||||
return copy;
|
||||
}
|
||||
|
||||
/**
|
||||
* Convert User entity to AdminUserSummary DTO, excluding sensitive fields like password and
|
||||
* apiKey.
|
||||
* Convert a User to AdminUserSummary (excludes sensitive fields); portal access is passed in.
|
||||
*/
|
||||
private AdminUserSummary convertUserToSummary(User user, Set<Long> leaderUserIds) {
|
||||
private AdminUserSummary convertUserToSummary(
|
||||
User user, Set<Long> leaderUserIds, Set<Long> portalAccessUserIds) {
|
||||
AdminUserSummary summary = new AdminUserSummary();
|
||||
summary.setId(user.getId());
|
||||
summary.setTeamLead(leaderUserIds.contains(user.getId()));
|
||||
// Authoritative portal access, same call /me uses, so the roster honors the configured
|
||||
// policy instead of the frontend guessing from role/team-leadership.
|
||||
summary.setPortalAccess(resourceAccessService.canAccessPortal(user));
|
||||
// Portal access (same policy /me uses).
|
||||
summary.setPortalAccess(portalAccessUserIds.contains(user.getId()));
|
||||
summary.setUsername(user.getUsername());
|
||||
summary.setEmail(user.getUsername()); // Use username as email for consistency
|
||||
summary.setRoleName(user.getRoleName());
|
||||
|
||||
+25
@@ -0,0 +1,25 @@
|
||||
package stirling.software.proprietary.integration.api;
|
||||
|
||||
/**
|
||||
* How an {@link stirling.software.proprietary.integration.model.IntegrationType#API} connection
|
||||
* authenticates.
|
||||
*/
|
||||
public enum ApiAuthType {
|
||||
/** No credentials; the endpoint is open or authorises by network position. */
|
||||
NONE,
|
||||
/** {@code Authorization: Bearer <token>}. */
|
||||
BEARER,
|
||||
/** {@code Authorization: Basic base64(username:password)}. */
|
||||
BASIC,
|
||||
/** The token in a caller-named header, e.g. {@code X-API-Key: <token>}. */
|
||||
HEADER,
|
||||
/**
|
||||
* The connection logs in first and reuses the short-lived token it gets back.
|
||||
*
|
||||
* <p>For the large class of enterprise APIs - ConsignO Cloud, OAuth2 client-credentials, and
|
||||
* others - where credentials buy a token rather than authenticating a call directly. Without
|
||||
* this a step could not reach them at all: each call needs a token, and a stateless step has
|
||||
* nowhere to obtain or keep one. See {@link ApiTokenLogin}.
|
||||
*/
|
||||
TOKEN_LOGIN
|
||||
}
|
||||
+130
@@ -0,0 +1,130 @@
|
||||
package stirling.software.proprietary.integration.api;
|
||||
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.Map;
|
||||
|
||||
import org.springframework.security.core.Authentication;
|
||||
import org.springframework.security.core.context.SecurityContextHolder;
|
||||
import org.springframework.security.core.userdetails.UserDetails;
|
||||
import org.springframework.stereotype.Service;
|
||||
import org.springframework.transaction.annotation.Transactional;
|
||||
|
||||
import lombok.RequiredArgsConstructor;
|
||||
|
||||
import stirling.software.proprietary.access.model.ResourceType;
|
||||
import stirling.software.proprietary.access.service.OwnershipService;
|
||||
import stirling.software.proprietary.integration.model.IntegrationConfig;
|
||||
import stirling.software.proprietary.integration.model.IntegrationType;
|
||||
import stirling.software.proprietary.integration.repository.IntegrationConfigRepository;
|
||||
import stirling.software.proprietary.security.model.User;
|
||||
import stirling.software.proprietary.security.service.UserService;
|
||||
|
||||
import tools.jackson.core.type.TypeReference;
|
||||
import tools.jackson.databind.ObjectMapper;
|
||||
|
||||
/**
|
||||
* Dereferences a step's {@code connectionId} to a stored integration config.
|
||||
*
|
||||
* <p>Mirrors {@code S3ConnectionResolver}. When an authenticated caller is present the connection
|
||||
* must be usable by them; a background worker thread carries no {@code SecurityContext} and skips
|
||||
* that check, relying on the step having been access-checked when the policy was saved or when an
|
||||
* ad-hoc run was dispatched - see {@link IntegrationStepValidator}, which is what makes that
|
||||
* assumption true rather than merely hoped for.
|
||||
*/
|
||||
@Service
|
||||
@RequiredArgsConstructor
|
||||
@Transactional(readOnly = true)
|
||||
public class ApiConnectionResolver {
|
||||
|
||||
private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper();
|
||||
|
||||
private final IntegrationConfigRepository connections;
|
||||
private final OwnershipService ownership;
|
||||
private final UserService userService;
|
||||
|
||||
/** The raw config map for a connection of the given type. */
|
||||
public Map<String, Object> resolveConfig(Long connectionId, IntegrationType type) {
|
||||
IntegrationConfig connection =
|
||||
connections
|
||||
.findById(connectionId)
|
||||
.filter(cfg -> cfg.getIntegrationType() == type)
|
||||
.filter(this::usableByCurrentUser)
|
||||
// Existence and access collapse into one error so a caller cannot tell
|
||||
// "no such connection" from "someone else's connection" and enumerate ids.
|
||||
.orElseThrow(
|
||||
() ->
|
||||
new IllegalArgumentException(
|
||||
"unknown or inaccessible "
|
||||
+ type.name().toLowerCase()
|
||||
+ " connection"));
|
||||
if (!connection.isEnabled()) {
|
||||
throw new IllegalArgumentException(
|
||||
type.name().toLowerCase() + " connection is disabled");
|
||||
}
|
||||
return configOf(connection);
|
||||
}
|
||||
|
||||
/** The settings for a generic {@code API} connection. */
|
||||
public ApiConnectionSettings resolve(Long connectionId) {
|
||||
return ApiConnectionSettings.from(resolveConfig(connectionId, IntegrationType.API));
|
||||
}
|
||||
|
||||
/** Parse a {@code connectionId} step parameter; null when absent. */
|
||||
public static Long connectionId(Object reference) {
|
||||
if (reference == null || (reference instanceof String s && s.isBlank())) {
|
||||
return null;
|
||||
}
|
||||
if (reference instanceof Number number) {
|
||||
return number.longValue();
|
||||
}
|
||||
try {
|
||||
return Long.valueOf(reference.toString().trim());
|
||||
} catch (NumberFormatException e) {
|
||||
throw new IllegalArgumentException(
|
||||
"'connectionId' is not a valid connection reference: " + reference);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether the current caller may use this connection. A missing principal means a worker
|
||||
* thread, where access was established earlier; it must never be the only thing standing
|
||||
* between a caller and a connection, or the check becomes a confused deputy.
|
||||
*/
|
||||
private boolean usableByCurrentUser(IntegrationConfig connection) {
|
||||
User user = currentUser();
|
||||
return user == null || ownership.canUse(ResourceType.INTEGRATION_CONFIG, connection, user);
|
||||
}
|
||||
|
||||
// Mirrors ResourceAccessSecurity's principal resolution; null when unauthenticated.
|
||||
private User currentUser() {
|
||||
Authentication auth = SecurityContextHolder.getContext().getAuthentication();
|
||||
if (auth == null || !auth.isAuthenticated()) {
|
||||
return null;
|
||||
}
|
||||
Object principal = auth.getPrincipal();
|
||||
if (principal instanceof User user) {
|
||||
return user;
|
||||
}
|
||||
if (principal instanceof UserDetails userDetails) {
|
||||
return userService.findByUsername(userDetails.getUsername()).orElse(null);
|
||||
}
|
||||
if (principal instanceof String username && !"anonymousUser".equals(username)) {
|
||||
return userService.findByUsername(username).orElse(null);
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
private static Map<String, Object> configOf(IntegrationConfig connection) {
|
||||
String json = connection.getConfig();
|
||||
if (json == null || json.isBlank()) {
|
||||
return Map.of();
|
||||
}
|
||||
try {
|
||||
return OBJECT_MAPPER.readValue(
|
||||
json, new TypeReference<LinkedHashMap<String, Object>>() {});
|
||||
} catch (Exception e) {
|
||||
throw new IllegalArgumentException(
|
||||
"connection '" + connection.getName() + "' has unreadable config", e);
|
||||
}
|
||||
}
|
||||
}
|
||||
+282
@@ -0,0 +1,282 @@
|
||||
package stirling.software.proprietary.integration.api;
|
||||
|
||||
import java.net.URI;
|
||||
import java.net.URISyntaxException;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.Locale;
|
||||
import java.util.Map;
|
||||
import java.util.Objects;
|
||||
import java.util.Set;
|
||||
|
||||
/**
|
||||
* A resolved {@code API} connection: where to call, and how to authenticate.
|
||||
*
|
||||
* <p>{@code baseUrl} is the security anchor of the whole feature. It is set by whoever can manage
|
||||
* the connection (an admin or team owner) and is the only thing that decides which host is
|
||||
* contacted. A pipeline step supplies a <em>relative path</em> only, resolved under this base by
|
||||
* {@link ExternalApiPaths}, so a step author can never pivot the call to a host of their choosing.
|
||||
* Widening that - letting a step pass a full URL - would turn every policy into an SSRF primitive.
|
||||
*
|
||||
* <p>Whether the base URL may resolve to a private address is deliberately <em>not</em> a field
|
||||
* here. Any user may create an API connection (unlike S3, which {@code IntegrationConfigService}
|
||||
* restricts to admins), so a per-connection opt-in would let a user grant themselves a fetch of the
|
||||
* cloud metadata service. It is an operator property instead - {@code
|
||||
* policies.allowPrivateApiEndpoints} - checked by {@link ApiIntegrationValidator}.
|
||||
*/
|
||||
public record ApiConnectionSettings(
|
||||
String baseUrl,
|
||||
ApiAuthType authType,
|
||||
String headerName,
|
||||
String headerPrefix,
|
||||
String token,
|
||||
String username,
|
||||
String password,
|
||||
Map<String, String> headers,
|
||||
ApiTokenLogin tokenLogin,
|
||||
Set<String> resultUrlHosts,
|
||||
int timeoutSeconds) {
|
||||
|
||||
static final String BASE_URL_OPTION = "baseUrl";
|
||||
static final String AUTH_TYPE_OPTION = "authType";
|
||||
static final String HEADER_NAME_OPTION = "headerName";
|
||||
static final String HEADER_PREFIX_OPTION = "headerPrefix";
|
||||
// "token"/"password" contain SecretMasker hints, so they mask on read and merge on update.
|
||||
static final String TOKEN_OPTION = "token";
|
||||
static final String USERNAME_OPTION = "username";
|
||||
static final String PASSWORD_OPTION = "password";
|
||||
static final String HEADERS_OPTION = "headers";
|
||||
static final String RESULT_URL_HOSTS_OPTION = "resultUrlHosts";
|
||||
static final String TIMEOUT_SECONDS_OPTION = "timeoutSeconds";
|
||||
|
||||
static final int DEFAULT_TIMEOUT_SECONDS = 60;
|
||||
private static final int MAX_TIMEOUT_SECONDS = 600;
|
||||
|
||||
public ApiConnectionSettings {
|
||||
headers = headers == null ? Map.of() : Map.copyOf(headers);
|
||||
resultUrlHosts = resultUrlHosts == null ? Set.of() : Set.copyOf(resultUrlHosts);
|
||||
}
|
||||
|
||||
/**
|
||||
* @throws IllegalArgumentException if the config is unusable; the message is surfaced to the
|
||||
* operator editing the connection, so it names the offending option.
|
||||
*/
|
||||
public static ApiConnectionSettings from(Map<String, Object> options) {
|
||||
String baseUrl = trimmed(options.get(BASE_URL_OPTION));
|
||||
if (baseUrl == null) {
|
||||
throw new IllegalArgumentException("api config requires a 'baseUrl' option");
|
||||
}
|
||||
URI uri = parseHttpUrl(baseUrl);
|
||||
if (uri.getQuery() != null || uri.getFragment() != null) {
|
||||
throw new IllegalArgumentException(
|
||||
"api config 'baseUrl' must not carry a query string or fragment");
|
||||
}
|
||||
|
||||
ApiAuthType authType = parseAuthType(trimmed(options.get(AUTH_TYPE_OPTION)));
|
||||
String headerName = trimmed(options.get(HEADER_NAME_OPTION));
|
||||
// Many APIs want a scheme before the token ("Authorization: Token abc",
|
||||
// "Authorization: DeepL-Auth-Key abc"). Without this a preset would have to make the
|
||||
// operator paste the scheme into the secret itself, which reads as a typo waiting to
|
||||
// happen.
|
||||
String headerPrefix = trimmed(options.get(HEADER_PREFIX_OPTION));
|
||||
String token = trimmed(options.get(TOKEN_OPTION));
|
||||
String username = trimmed(options.get(USERNAME_OPTION));
|
||||
String password = trimmed(options.get(PASSWORD_OPTION));
|
||||
|
||||
switch (authType) {
|
||||
case BEARER -> require(token, "api config authType 'BEARER' requires a 'token'");
|
||||
case HEADER -> {
|
||||
require(token, "api config authType 'HEADER' requires a 'token'");
|
||||
require(headerName, "api config authType 'HEADER' requires a 'headerName'");
|
||||
if (!ExternalApiHeaders.isValidName(headerName)) {
|
||||
throw new IllegalArgumentException(
|
||||
"api config 'headerName' is not a valid HTTP header name: "
|
||||
+ headerName);
|
||||
}
|
||||
}
|
||||
case BASIC -> {
|
||||
require(username, "api config authType 'BASIC' requires a 'username'");
|
||||
require(password, "api config authType 'BASIC' requires a 'password'");
|
||||
}
|
||||
case TOKEN_LOGIN -> {
|
||||
/* validated by ApiTokenLogin.from below */
|
||||
}
|
||||
case NONE -> {
|
||||
/* nothing to check */
|
||||
}
|
||||
}
|
||||
|
||||
return new ApiConnectionSettings(
|
||||
stripTrailingSlash(baseUrl),
|
||||
authType,
|
||||
headerName,
|
||||
headerPrefix,
|
||||
token,
|
||||
username,
|
||||
password,
|
||||
parseHeaders(options.get(HEADERS_OPTION)),
|
||||
authType == ApiAuthType.TOKEN_LOGIN ? ApiTokenLogin.from(options) : null,
|
||||
parseResultUrlHosts(options.get(RESULT_URL_HOSTS_OPTION)),
|
||||
parseTimeout(options.get(TIMEOUT_SECONDS_OPTION)));
|
||||
}
|
||||
|
||||
/** The configured base as a URI; callers resolve step paths under it. */
|
||||
public URI baseUri() {
|
||||
return URI.create(baseUrl);
|
||||
}
|
||||
|
||||
/**
|
||||
* Identity of this connection's login for token-cache purposes. Includes the credentials, so
|
||||
* editing a password evicts the token cached under the old one rather than reusing it until it
|
||||
* expires.
|
||||
*/
|
||||
String tokenCacheKey() {
|
||||
return baseUrl + "|" + Objects.hash(tokenLogin);
|
||||
}
|
||||
|
||||
private static URI parseHttpUrl(String value) {
|
||||
URI uri;
|
||||
try {
|
||||
uri = new URI(value);
|
||||
} catch (URISyntaxException e) {
|
||||
throw new IllegalArgumentException("api config 'baseUrl' is not a valid URL", e);
|
||||
}
|
||||
String scheme = uri.getScheme() == null ? null : uri.getScheme().toLowerCase(Locale.ROOT);
|
||||
if (!"http".equals(scheme) && !"https".equals(scheme)) {
|
||||
throw new IllegalArgumentException(
|
||||
"api config 'baseUrl' must be an http(s) URL, e.g. https://api.example.com");
|
||||
}
|
||||
if (uri.getHost() == null || uri.getHost().isBlank()) {
|
||||
throw new IllegalArgumentException("api config 'baseUrl' must include a host");
|
||||
}
|
||||
return uri;
|
||||
}
|
||||
|
||||
private static ApiAuthType parseAuthType(String value) {
|
||||
if (value == null) {
|
||||
return ApiAuthType.NONE;
|
||||
}
|
||||
try {
|
||||
return ApiAuthType.valueOf(value.toUpperCase(Locale.ROOT));
|
||||
} catch (IllegalArgumentException e) {
|
||||
throw new IllegalArgumentException(
|
||||
"api config 'authType' must be one of NONE, BEARER, BASIC, HEADER; got "
|
||||
+ value);
|
||||
}
|
||||
}
|
||||
|
||||
/** Static headers sent on every call. Rejects anything auth-bearing to keep one auth path. */
|
||||
private static Map<String, String> parseHeaders(Object value) {
|
||||
if (value == null) {
|
||||
return Map.of();
|
||||
}
|
||||
if (!(value instanceof Map<?, ?> raw)) {
|
||||
throw new IllegalArgumentException("api config 'headers' must be an object");
|
||||
}
|
||||
Map<String, String> headers = new LinkedHashMap<>();
|
||||
for (Map.Entry<?, ?> entry : raw.entrySet()) {
|
||||
String name = trimmed(entry.getKey());
|
||||
String headerValue = entry.getValue() == null ? null : entry.getValue().toString();
|
||||
if (name == null) {
|
||||
continue;
|
||||
}
|
||||
if (!ExternalApiHeaders.isValidName(name)) {
|
||||
throw new IllegalArgumentException(
|
||||
"api config 'headers' has an invalid header name: " + name);
|
||||
}
|
||||
if (ExternalApiHeaders.isReserved(name)) {
|
||||
throw new IllegalArgumentException(
|
||||
"api config 'headers' must not set '"
|
||||
+ name
|
||||
+ "'; use 'authType' and 'token' instead");
|
||||
}
|
||||
if (headerValue == null || !ExternalApiHeaders.isValidValue(headerValue)) {
|
||||
throw new IllegalArgumentException(
|
||||
"api config 'headers' has an invalid value for '" + name + "'");
|
||||
}
|
||||
headers.put(name, headerValue);
|
||||
}
|
||||
return headers;
|
||||
}
|
||||
|
||||
/**
|
||||
* Hosts a result may be fetched from, beyond the connection's own. Declared by the operator
|
||||
* because the alternative - trusting the host named in the API's response - is an SSRF.
|
||||
*/
|
||||
private static Set<String> parseResultUrlHosts(Object value) {
|
||||
if (value == null) {
|
||||
return Set.of();
|
||||
}
|
||||
if (!(value instanceof java.util.List<?> list)) {
|
||||
throw new IllegalArgumentException(
|
||||
"api config 'resultUrlHosts' must be a list of hostnames");
|
||||
}
|
||||
Set<String> out = new java.util.LinkedHashSet<>();
|
||||
for (Object entry : list) {
|
||||
String host = trimmed(entry);
|
||||
if (host == null) {
|
||||
continue;
|
||||
}
|
||||
if (host.contains("/") || host.contains(":") || host.contains("*")) {
|
||||
// A URL, port or wildcard here would read as broader than it is; subdomains are
|
||||
// already covered by the "endsWith('.' + host)" rule at match time.
|
||||
throw new IllegalArgumentException(
|
||||
"api config 'resultUrlHosts' takes bare hostnames, e.g."
|
||||
+ " cdn.vendor.com; got "
|
||||
+ host);
|
||||
}
|
||||
out.add(host.toLowerCase(Locale.ROOT));
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
private static int parseTimeout(Object value) {
|
||||
if (value == null) {
|
||||
return DEFAULT_TIMEOUT_SECONDS;
|
||||
}
|
||||
int seconds;
|
||||
try {
|
||||
seconds = Integer.parseInt(value.toString().trim());
|
||||
} catch (NumberFormatException e) {
|
||||
throw new IllegalArgumentException("api config 'timeoutSeconds' must be a number");
|
||||
}
|
||||
if (seconds < 1 || seconds > MAX_TIMEOUT_SECONDS) {
|
||||
throw new IllegalArgumentException(
|
||||
"api config 'timeoutSeconds' must be between 1 and " + MAX_TIMEOUT_SECONDS);
|
||||
}
|
||||
return seconds;
|
||||
}
|
||||
|
||||
private static void require(String value, String message) {
|
||||
if (value == null) {
|
||||
throw new IllegalArgumentException(message);
|
||||
}
|
||||
}
|
||||
|
||||
private static String stripTrailingSlash(String value) {
|
||||
String out = value;
|
||||
while (out.endsWith("/")) {
|
||||
out = out.substring(0, out.length() - 1);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
private static String trimmed(Object value) {
|
||||
if (value == null) {
|
||||
return null;
|
||||
}
|
||||
String text = value.toString().trim();
|
||||
return text.isEmpty() ? null : text;
|
||||
}
|
||||
|
||||
/** Never prints the credentials, so an accidental log line cannot leak them. */
|
||||
@Override
|
||||
public String toString() {
|
||||
return "ApiConnectionSettings[baseUrl="
|
||||
+ baseUrl
|
||||
+ ", authType="
|
||||
+ authType
|
||||
+ ", timeoutSeconds="
|
||||
+ timeoutSeconds
|
||||
+ "]";
|
||||
}
|
||||
}
|
||||
+109
@@ -0,0 +1,109 @@
|
||||
package stirling.software.proprietary.integration.api;
|
||||
|
||||
import java.util.Map;
|
||||
|
||||
import org.springframework.stereotype.Component;
|
||||
|
||||
import lombok.RequiredArgsConstructor;
|
||||
|
||||
import stirling.software.common.model.ApplicationProperties;
|
||||
import stirling.software.proprietary.cluster.s3.S3Clients;
|
||||
import stirling.software.proprietary.integration.model.IntegrationType;
|
||||
import stirling.software.proprietary.integration.service.IntegrationConfigValidator;
|
||||
|
||||
/**
|
||||
* The {@code API} connection schema, enforced when the config is saved: an http(s) base URL, a
|
||||
* coherent auth block, and a host that must not reach private addresses without the operator
|
||||
* opt-in.
|
||||
*
|
||||
* <p>The host check runs here so a bad connection fails in the form rather than mid-run. It is not
|
||||
* the only check - {@link ExternalApiCaller} re-checks before dispatch, because DNS can be
|
||||
* re-pointed at a private address long after save time (a check-then-use gap this validator alone
|
||||
* cannot close).
|
||||
*/
|
||||
@Component
|
||||
@RequiredArgsConstructor
|
||||
public class ApiIntegrationValidator implements IntegrationConfigValidator {
|
||||
|
||||
private final ApplicationProperties applicationProperties;
|
||||
|
||||
@Override
|
||||
public IntegrationType type() {
|
||||
return IntegrationType.API;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void validate(Map<String, Object> config) {
|
||||
ApiConnectionSettings settings = ApiConnectionSettings.from(config);
|
||||
requirePublicHost(settings, applicationProperties, "API connection base URL");
|
||||
}
|
||||
|
||||
/**
|
||||
* Shared by every integration type that dials an operator-supplied host, so they cannot drift
|
||||
* apart on what counts as reachable.
|
||||
*/
|
||||
static void requirePublicHost(
|
||||
ApiConnectionSettings settings,
|
||||
ApplicationProperties applicationProperties,
|
||||
String settingName) {
|
||||
// Block the cloud metadata service unconditionally - before the opt-in check. The private-
|
||||
// endpoint opt-in exists for on-prem services (RFC1918, an internal gateway), but the
|
||||
// metadata endpoint is never a real integration and reaching it is the highest-value SSRF:
|
||||
// it hands out the instance's own IAM credentials. So it stays blocked even when the
|
||||
// operator has allowed private endpoints.
|
||||
denyCloudMetadata(settings.baseUri(), settingName);
|
||||
try {
|
||||
S3Clients.validateEndpointHost(
|
||||
settings.baseUri(),
|
||||
applicationProperties.getPolicies().isAllowPrivateApiEndpoints(),
|
||||
settingName,
|
||||
"set policies.allowPrivateApiEndpoints=true to opt in (e.g. for an on-prem"
|
||||
+ " integration).");
|
||||
} catch (IllegalStateException e) {
|
||||
throw new IllegalArgumentException(e.getMessage(), e);
|
||||
}
|
||||
}
|
||||
|
||||
/** AWS/GCP/Azure, Oracle and IBM metadata addresses; mirrors {@code SsrfProtectionService}. */
|
||||
private static final java.util.Set<String> CLOUD_METADATA_IPS =
|
||||
java.util.Set.of(
|
||||
"169.254.169.254", "169.254.169.253", "169.254.169.250", "fd00:ec2::254");
|
||||
|
||||
private static void denyCloudMetadata(java.net.URI uri, String settingName) {
|
||||
String host = uri.getHost();
|
||||
if (host == null || host.isBlank()) {
|
||||
return; // a missing host is S3Clients' error to report, with its own message
|
||||
}
|
||||
java.net.InetAddress[] addresses;
|
||||
try {
|
||||
addresses = java.net.InetAddress.getAllByName(host);
|
||||
} catch (java.net.UnknownHostException e) {
|
||||
return; // an unresolvable host is likewise left to S3Clients to reject
|
||||
}
|
||||
for (java.net.InetAddress address : addresses) {
|
||||
String ip = normalise(address.getHostAddress());
|
||||
if (CLOUD_METADATA_IPS.stream().anyMatch(ip::startsWith)) {
|
||||
throw new IllegalArgumentException(
|
||||
settingName
|
||||
+ " host '"
|
||||
+ host
|
||||
+ "' resolves to the cloud metadata service ("
|
||||
+ ip
|
||||
+ "), which is never a valid integration target.");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** Strip an IPv4-mapped-IPv6 prefix and any zone id so the compare sees a bare address. */
|
||||
private static String normalise(String ip) {
|
||||
String out = ip;
|
||||
int zone = out.indexOf('%');
|
||||
if (zone >= 0) {
|
||||
out = out.substring(0, zone);
|
||||
}
|
||||
if (out.startsWith("::ffff:")) {
|
||||
out = out.substring(7);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
}
|
||||
+149
@@ -0,0 +1,149 @@
|
||||
package stirling.software.proprietary.integration.api;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.net.URI;
|
||||
import java.net.http.HttpClient;
|
||||
import java.net.http.HttpRequest;
|
||||
import java.time.Duration;
|
||||
import java.util.Map;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
|
||||
import org.springframework.http.MediaType;
|
||||
|
||||
import com.github.benmanes.caffeine.cache.Cache;
|
||||
import com.github.benmanes.caffeine.cache.Caffeine;
|
||||
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
import tools.jackson.databind.ObjectMapper;
|
||||
|
||||
/**
|
||||
* Obtains and caches the short-lived tokens of {@link ApiAuthType#TOKEN_LOGIN} connections.
|
||||
*
|
||||
* <p>The step that uses a token is stateless and runs once per document, so without a cache a
|
||||
* hundred-document policy would perform a hundred logins - which many vendors rate-limit, and some
|
||||
* treat as suspicious. The cache is keyed on the connection's login identity (credentials included)
|
||||
* so that editing a password does not keep reusing the token bought with the old one.
|
||||
*
|
||||
* <p>Entries expire well inside the vendor's stated lifetime, and a 401 additionally evicts and
|
||||
* retries once ({@link ExternalApiCaller}), so a token that expires early - or is revoked - costs
|
||||
* one retry rather than a failed run.
|
||||
*/
|
||||
@Slf4j
|
||||
public class ApiTokenCache {
|
||||
|
||||
/** Bounded so a deployment with many connections cannot grow this without limit. */
|
||||
private static final int MAX_ENTRIES = 500;
|
||||
|
||||
private final Cache<String, String> tokens;
|
||||
private final HttpClient httpClient;
|
||||
private final ObjectMapper objectMapper;
|
||||
|
||||
ApiTokenCache(HttpClient httpClient, ObjectMapper objectMapper) {
|
||||
this.httpClient = httpClient;
|
||||
this.objectMapper = objectMapper;
|
||||
this.tokens =
|
||||
Caffeine.newBuilder()
|
||||
.maximumSize(MAX_ENTRIES)
|
||||
// Per-entry, because each connection states its own lifetime.
|
||||
.expireAfter(
|
||||
new com.github.benmanes.caffeine.cache.Expiry<String, String>() {
|
||||
@Override
|
||||
public long expireAfterCreate(
|
||||
String key, String value, long currentTime) {
|
||||
return ttlNanos(key);
|
||||
}
|
||||
|
||||
@Override
|
||||
public long expireAfterUpdate(
|
||||
String key,
|
||||
String value,
|
||||
long currentTime,
|
||||
long currentDuration) {
|
||||
return ttlNanos(key);
|
||||
}
|
||||
|
||||
@Override
|
||||
public long expireAfterRead(
|
||||
String key,
|
||||
String value,
|
||||
long currentTime,
|
||||
long currentDuration) {
|
||||
// Reading must not extend a token's life: the vendor's
|
||||
// clock is running regardless of how often we use it.
|
||||
return currentDuration;
|
||||
}
|
||||
})
|
||||
.build();
|
||||
}
|
||||
|
||||
// The TTL travels in the key so the Expiry callbacks can see it without a second lookup.
|
||||
private static long ttlNanos(String key) {
|
||||
int seconds = Integer.parseInt(key.substring(key.lastIndexOf('#') + 1));
|
||||
return TimeUnit.SECONDS.toNanos(seconds);
|
||||
}
|
||||
|
||||
/**
|
||||
* The connection's current token, logging in if there is not a live one.
|
||||
*
|
||||
* @throws IOException if the login call fails or returns no token
|
||||
*/
|
||||
String token(ApiConnectionSettings settings) throws IOException {
|
||||
String key = cacheKey(settings);
|
||||
String cached = tokens.getIfPresent(key);
|
||||
if (cached != null) {
|
||||
return cached;
|
||||
}
|
||||
String token = login(settings);
|
||||
tokens.put(key, token);
|
||||
return token;
|
||||
}
|
||||
|
||||
/** Drop the cached token, e.g. after a 401 says it is no longer accepted. */
|
||||
void invalidate(ApiConnectionSettings settings) {
|
||||
tokens.invalidate(cacheKey(settings));
|
||||
}
|
||||
|
||||
private static String cacheKey(ApiConnectionSettings settings) {
|
||||
return settings.tokenCacheKey() + "#" + settings.tokenLogin().tokenTtlSeconds();
|
||||
}
|
||||
|
||||
private String login(ApiConnectionSettings settings) throws IOException {
|
||||
ApiTokenLogin login = settings.tokenLogin();
|
||||
URI target = ExternalApiPaths.resolve(settings.baseUri(), login.loginPath());
|
||||
|
||||
HttpRequest.Builder request =
|
||||
HttpRequest.newBuilder(target)
|
||||
.timeout(Duration.ofSeconds(settings.timeoutSeconds()))
|
||||
.header("Content-Type", MediaType.APPLICATION_JSON_VALUE)
|
||||
.POST(
|
||||
HttpRequest.BodyPublishers.ofByteArray(
|
||||
objectMapper.writeValueAsBytes(login.loginBody())));
|
||||
login.loginHeaders().forEach(request::header);
|
||||
|
||||
ExternalApiCaller.Response response =
|
||||
ExternalApiCaller.send(httpClient, request.build(), target);
|
||||
if (!response.isSuccess()) {
|
||||
// Deliberately does not echo the body: a login failure response can repeat the
|
||||
// credentials back, and this message reaches the run log.
|
||||
throw new IOException(
|
||||
"Login to "
|
||||
+ target.getHost()
|
||||
+ login.loginPath()
|
||||
+ " returned HTTP "
|
||||
+ response.status());
|
||||
}
|
||||
try {
|
||||
String token = login.extractToken(response, objectMapper);
|
||||
log.debug("[external-api] obtained a token from {}", target.getHost());
|
||||
return token;
|
||||
} catch (IllegalStateException e) {
|
||||
throw new IOException(e.getMessage(), e);
|
||||
}
|
||||
}
|
||||
|
||||
/** The auth header for an authenticated call. */
|
||||
Map.Entry<String, String> authHeader(ApiConnectionSettings settings) throws IOException {
|
||||
return settings.tokenLogin().authHeader(token(settings));
|
||||
}
|
||||
}
|
||||
+209
@@ -0,0 +1,209 @@
|
||||
package stirling.software.proprietary.integration.api;
|
||||
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.Map;
|
||||
|
||||
import tools.jackson.databind.JsonNode;
|
||||
import tools.jackson.databind.ObjectMapper;
|
||||
|
||||
/**
|
||||
* How a connection turns credentials into a short-lived token.
|
||||
*
|
||||
* <p>Modelled on what real APIs actually do rather than on one vendor. The two axes that vary are
|
||||
* where the token comes back ({@code tokenResponseHeader} or {@code tokenResponseJsonPath}) and how
|
||||
* it is then presented ({@code tokenHeaderName} + {@code tokenPrefix}). That covers both ends of
|
||||
* the spectrum:
|
||||
*
|
||||
* <ul>
|
||||
* <li>ConsignO Cloud - {@code POST /auth/login} with {@code X-Client-Id}/{@code X-Client-Secret}
|
||||
* headers and a {@code {username, password, tenantId}} body, returning the token in the
|
||||
* {@code X-Auth-Token} <em>response header</em>, which is then sent back as {@code
|
||||
* X-Auth-Token}.
|
||||
* <li>OAuth2 client-credentials - a form or JSON post returning {@code {"access_token": ...}} in
|
||||
* the body, sent back as {@code Authorization: Bearer ...}.
|
||||
* </ul>
|
||||
*
|
||||
* <p>{@code loginBody} and {@code loginHeaders} are stored as nested maps rather than a
|
||||
* pre-rendered JSON string so {@code SecretMasker} can recurse and mask the {@code password} /
|
||||
* {@code X-Client-Secret} entries inside them. A flat string would sail past it and hand the
|
||||
* password back in plaintext on every read of the connection.
|
||||
*/
|
||||
record ApiTokenLogin(
|
||||
String loginPath,
|
||||
Map<String, Object> loginBody,
|
||||
Map<String, String> loginHeaders,
|
||||
String tokenResponseHeader,
|
||||
String tokenResponseJsonPath,
|
||||
String tokenHeaderName,
|
||||
String tokenPrefix,
|
||||
int tokenTtlSeconds) {
|
||||
|
||||
static final String LOGIN_PATH_OPTION = "loginPath";
|
||||
static final String LOGIN_BODY_OPTION = "loginBody";
|
||||
static final String LOGIN_HEADERS_OPTION = "loginHeaders";
|
||||
static final String TOKEN_RESPONSE_HEADER_OPTION = "tokenResponseHeader";
|
||||
static final String TOKEN_RESPONSE_JSON_PATH_OPTION = "tokenResponseJsonPath";
|
||||
static final String TOKEN_HEADER_NAME_OPTION = "tokenHeaderName";
|
||||
static final String TOKEN_PREFIX_OPTION = "tokenPrefix";
|
||||
static final String TOKEN_TTL_SECONDS_OPTION = "tokenTtlSeconds";
|
||||
|
||||
/**
|
||||
* Conservative default. ConsignO's token lasts 30 minutes; caching for 25 leaves room for a
|
||||
* slow call to finish on a token that was still valid when it started. A cache that expired
|
||||
* exactly on the vendor's boundary would fail intermittently and look like a network fault.
|
||||
*/
|
||||
static final int DEFAULT_TOKEN_TTL_SECONDS = 1500;
|
||||
|
||||
private static final int MAX_TOKEN_TTL_SECONDS = 86400;
|
||||
|
||||
ApiTokenLogin {
|
||||
loginBody = loginBody == null ? Map.of() : Map.copyOf(loginBody);
|
||||
loginHeaders = loginHeaders == null ? Map.of() : Map.copyOf(loginHeaders);
|
||||
}
|
||||
|
||||
static ApiTokenLogin from(Map<String, Object> options) {
|
||||
String loginPath = trimmed(options.get(LOGIN_PATH_OPTION));
|
||||
if (loginPath == null) {
|
||||
throw new IllegalArgumentException(
|
||||
"api config authType 'TOKEN_LOGIN' requires a 'loginPath', e.g. /auth/login");
|
||||
}
|
||||
String responseHeader = trimmed(options.get(TOKEN_RESPONSE_HEADER_OPTION));
|
||||
String responseJsonPath = trimmed(options.get(TOKEN_RESPONSE_JSON_PATH_OPTION));
|
||||
if ((responseHeader == null) == (responseJsonPath == null)) {
|
||||
throw new IllegalArgumentException(
|
||||
"api config authType 'TOKEN_LOGIN' needs exactly one of"
|
||||
+ " 'tokenResponseHeader' (e.g. X-Auth-Token) or"
|
||||
+ " 'tokenResponseJsonPath' (e.g. access_token) to say where the token"
|
||||
+ " comes back");
|
||||
}
|
||||
String tokenHeaderName = trimmed(options.get(TOKEN_HEADER_NAME_OPTION));
|
||||
if (tokenHeaderName == null) {
|
||||
throw new IllegalArgumentException(
|
||||
"api config authType 'TOKEN_LOGIN' requires a 'tokenHeaderName' saying which"
|
||||
+ " header carries the token back, e.g. X-Auth-Token or Authorization");
|
||||
}
|
||||
if (!ExternalApiHeaders.isValidName(tokenHeaderName)) {
|
||||
throw new IllegalArgumentException(
|
||||
"api config 'tokenHeaderName' is not a valid HTTP header name: "
|
||||
+ tokenHeaderName);
|
||||
}
|
||||
if (responseHeader != null && !ExternalApiHeaders.isValidName(responseHeader)) {
|
||||
throw new IllegalArgumentException(
|
||||
"api config 'tokenResponseHeader' is not a valid HTTP header name: "
|
||||
+ responseHeader);
|
||||
}
|
||||
|
||||
return new ApiTokenLogin(
|
||||
loginPath,
|
||||
nestedObject(options.get(LOGIN_BODY_OPTION), LOGIN_BODY_OPTION),
|
||||
loginHeaders(options.get(LOGIN_HEADERS_OPTION)),
|
||||
responseHeader,
|
||||
responseJsonPath,
|
||||
tokenHeaderName,
|
||||
trimmed(options.get(TOKEN_PREFIX_OPTION)) == null
|
||||
? ""
|
||||
: trimmed(options.get(TOKEN_PREFIX_OPTION)) + " ",
|
||||
ttl(options.get(TOKEN_TTL_SECONDS_OPTION)));
|
||||
}
|
||||
|
||||
/** Pull the token out of a login response. */
|
||||
String extractToken(ExternalApiCaller.Response response, ObjectMapper objectMapper) {
|
||||
if (tokenResponseHeader != null) {
|
||||
String value = response.header(tokenResponseHeader);
|
||||
if (value == null || value.isBlank()) {
|
||||
throw new IllegalStateException(
|
||||
"Login succeeded but returned no '"
|
||||
+ tokenResponseHeader
|
||||
+ "' response header");
|
||||
}
|
||||
return value;
|
||||
}
|
||||
JsonNode node = response.bodyAsJson(objectMapper);
|
||||
for (String segment : tokenResponseJsonPath.split("\\.")) {
|
||||
if (node == null) {
|
||||
break;
|
||||
}
|
||||
node = node.get(segment);
|
||||
}
|
||||
if (node == null || !node.isValueNode() || node.asString().isBlank()) {
|
||||
throw new IllegalStateException(
|
||||
"Login succeeded but its body had no token at '" + tokenResponseJsonPath + "'");
|
||||
}
|
||||
return node.asString();
|
||||
}
|
||||
|
||||
/** The header to send on an authenticated call. */
|
||||
Map.Entry<String, String> authHeader(String token) {
|
||||
return Map.entry(tokenHeaderName, tokenPrefix + token);
|
||||
}
|
||||
|
||||
private static Map<String, Object> nestedObject(Object value, String option) {
|
||||
if (value == null) {
|
||||
return Map.of();
|
||||
}
|
||||
if (!(value instanceof Map<?, ?> raw)) {
|
||||
throw new IllegalArgumentException("api config '" + option + "' must be an object");
|
||||
}
|
||||
Map<String, Object> out = new LinkedHashMap<>();
|
||||
raw.forEach((key, entry) -> out.put(String.valueOf(key), entry));
|
||||
return out;
|
||||
}
|
||||
|
||||
private static Map<String, String> loginHeaders(Object value) {
|
||||
Map<String, String> out = new LinkedHashMap<>();
|
||||
nestedObject(value, LOGIN_HEADERS_OPTION)
|
||||
.forEach(
|
||||
(name, entry) -> {
|
||||
String headerValue = entry == null ? null : entry.toString();
|
||||
if (!ExternalApiHeaders.isValidName(name)) {
|
||||
throw new IllegalArgumentException(
|
||||
"api config 'loginHeaders' has an invalid header name: "
|
||||
+ name);
|
||||
}
|
||||
if (headerValue == null
|
||||
|| !ExternalApiHeaders.isValidValue(headerValue)) {
|
||||
throw new IllegalArgumentException(
|
||||
"api config 'loginHeaders' has an invalid value for '"
|
||||
+ name
|
||||
+ "'");
|
||||
}
|
||||
out.put(name, headerValue);
|
||||
});
|
||||
return out;
|
||||
}
|
||||
|
||||
private static int ttl(Object value) {
|
||||
if (value == null) {
|
||||
return DEFAULT_TOKEN_TTL_SECONDS;
|
||||
}
|
||||
int seconds;
|
||||
try {
|
||||
seconds = Integer.parseInt(value.toString().trim());
|
||||
} catch (NumberFormatException e) {
|
||||
throw new IllegalArgumentException("api config 'tokenTtlSeconds' must be a number");
|
||||
}
|
||||
if (seconds < 1 || seconds > MAX_TOKEN_TTL_SECONDS) {
|
||||
throw new IllegalArgumentException(
|
||||
"api config 'tokenTtlSeconds' must be between 1 and " + MAX_TOKEN_TTL_SECONDS);
|
||||
}
|
||||
return seconds;
|
||||
}
|
||||
|
||||
private static String trimmed(Object value) {
|
||||
if (value == null) {
|
||||
return null;
|
||||
}
|
||||
String text = value.toString().trim();
|
||||
return text.isEmpty() ? null : text;
|
||||
}
|
||||
|
||||
/** Never prints the login body or headers: both carry the credentials. */
|
||||
@Override
|
||||
public String toString() {
|
||||
return "ApiTokenLogin[loginPath="
|
||||
+ loginPath
|
||||
+ ", tokenTtlSeconds="
|
||||
+ tokenTtlSeconds
|
||||
+ "]";
|
||||
}
|
||||
}
|
||||
+180
@@ -0,0 +1,180 @@
|
||||
package stirling.software.proprietary.integration.api;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.security.MessageDigest;
|
||||
import java.security.NoSuchAlgorithmException;
|
||||
import java.time.Instant;
|
||||
import java.util.Base64;
|
||||
import java.util.Calendar;
|
||||
import java.util.HexFormat;
|
||||
import java.util.List;
|
||||
import java.util.Locale;
|
||||
|
||||
import org.apache.pdfbox.Loader;
|
||||
import org.apache.pdfbox.pdmodel.PDDocument;
|
||||
import org.apache.pdfbox.pdmodel.PDDocumentInformation;
|
||||
import org.springframework.web.multipart.MultipartFile;
|
||||
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
import stirling.software.common.service.PdfMetadataService;
|
||||
import stirling.software.proprietary.integration.purview.PdfSensitivityLabels;
|
||||
import stirling.software.proprietary.integration.purview.SensitivityLabel;
|
||||
|
||||
import tools.jackson.databind.JsonNode;
|
||||
import tools.jackson.databind.ObjectMapper;
|
||||
import tools.jackson.databind.node.ObjectNode;
|
||||
|
||||
/**
|
||||
* Everything Stirling already knows about the document and the run, as one JSON object.
|
||||
*
|
||||
* <p>An external API almost always wants more than the bytes: what the file is, what it was called,
|
||||
* whether it is already classified or labelled, and which policy sent it. All of that is in hand at
|
||||
* the moment of the call, so it is offered rather than left for the operator to re-derive - most
|
||||
* usefully the Purview label and the classifier's verdict, which turn a call-out into something the
|
||||
* receiving system can make a decision with.
|
||||
*
|
||||
* <p>The shape is also the namespace for placeholders (see {@link Placeholders}), so {@code
|
||||
* {{document.sha256}}} or {@code {{sensitivityLabel.name}}} in a field, path, or header resolves
|
||||
* against exactly what is documented here:
|
||||
*
|
||||
* <pre>
|
||||
* document.filename | .extension | .contentType | .sizeBytes | .sha256 | .base64
|
||||
* .pageCount | .encrypted | .title | .author | .subject | .keywords
|
||||
* .creator | .producer | .created | .modified
|
||||
* classification.* the classifier policy's verdict, when it has run
|
||||
* sensitivityLabel.labelId | .name | .siteId | .method | .protected
|
||||
* run.policyName | .runId | .timestamp
|
||||
* </pre>
|
||||
*
|
||||
* <p>Every field is best-effort: a non-PDF, an unparseable PDF, or an ad-hoc run with no policy
|
||||
* simply omits what it cannot know. Building the context must never be the reason a step fails.
|
||||
*/
|
||||
@Slf4j
|
||||
final class DocumentContext {
|
||||
|
||||
private DocumentContext() {}
|
||||
|
||||
static ObjectNode build(
|
||||
MultipartFile file,
|
||||
byte[] content,
|
||||
String policyName,
|
||||
String runId,
|
||||
ObjectMapper objectMapper) {
|
||||
ObjectNode root = objectMapper.createObjectNode();
|
||||
ObjectNode document = root.putObject("document");
|
||||
|
||||
String filename = file.getOriginalFilename();
|
||||
document.put("filename", filename);
|
||||
document.put("extension", extensionOf(filename));
|
||||
document.put("contentType", file.getContentType());
|
||||
document.put("sizeBytes", content.length);
|
||||
document.put("sha256", sha256(content));
|
||||
// The bytes themselves, for steps that carry the document inside a JSON body
|
||||
// (an attachment field, a signing payload) rather than as multipart.
|
||||
document.put("base64", Base64.getEncoder().encodeToString(content));
|
||||
|
||||
if (looksLikePdf(content)) {
|
||||
addPdfFacts(document, root, content, objectMapper);
|
||||
}
|
||||
|
||||
ObjectNode run = root.putObject("run");
|
||||
run.put("policyName", policyName);
|
||||
run.put("runId", runId);
|
||||
run.put("timestamp", Instant.now().toString());
|
||||
return root;
|
||||
}
|
||||
|
||||
/** PDF-only facts. A document we cannot parse still gets the basics above. */
|
||||
private static void addPdfFacts(
|
||||
ObjectNode document, ObjectNode root, byte[] content, ObjectMapper objectMapper) {
|
||||
try (PDDocument pdf = Loader.loadPDF(content)) {
|
||||
document.put("pageCount", pdf.getNumberOfPages());
|
||||
document.put("encrypted", pdf.isEncrypted());
|
||||
|
||||
PDDocumentInformation info = pdf.getDocumentInformation();
|
||||
document.put("title", info.getTitle());
|
||||
document.put("author", info.getAuthor());
|
||||
document.put("subject", info.getSubject());
|
||||
document.put("keywords", info.getKeywords());
|
||||
document.put("creator", info.getCreator());
|
||||
document.put("producer", info.getProducer());
|
||||
document.put("created", toIso(info.getCreationDate()));
|
||||
document.put("modified", toIso(info.getModificationDate()));
|
||||
|
||||
addClassification(root, info, objectMapper);
|
||||
addSensitivityLabel(root, pdf);
|
||||
} catch (IOException | RuntimeException e) {
|
||||
// An encrypted or malformed PDF is a normal thing to send to an external API; the
|
||||
// extra facts are a convenience, not a precondition.
|
||||
log.debug("Could not read PDF facts for the step context: {}", e.getMessage());
|
||||
}
|
||||
}
|
||||
|
||||
/** The classifier policy's verdict, so a call-out can act on it without re-classifying. */
|
||||
private static void addClassification(
|
||||
ObjectNode root, PDDocumentInformation info, ObjectMapper objectMapper) {
|
||||
String raw = info.getCustomMetadataValue(PdfMetadataService.CLASSIFICATION_KEY);
|
||||
if (raw == null || raw.isBlank()) {
|
||||
return;
|
||||
}
|
||||
try {
|
||||
JsonNode parsed = objectMapper.readTree(raw);
|
||||
root.set("classification", parsed);
|
||||
} catch (RuntimeException e) {
|
||||
// Written by another tool; if it is not JSON, pass it through as text rather than drop
|
||||
// it - the receiving system may still recognise it.
|
||||
root.put("classification", raw);
|
||||
}
|
||||
}
|
||||
|
||||
/** The Purview label already on the document, if any. */
|
||||
private static void addSensitivityLabel(ObjectNode root, PDDocument pdf) {
|
||||
List<SensitivityLabel> labels = PdfSensitivityLabels.readAll(pdf);
|
||||
if (labels.isEmpty()) {
|
||||
return;
|
||||
}
|
||||
SensitivityLabel label = labels.get(0);
|
||||
ObjectNode node = root.putObject("sensitivityLabel");
|
||||
node.put("labelId", label.labelId());
|
||||
node.put("name", label.name());
|
||||
node.put("siteId", label.siteId());
|
||||
node.put("method", label.method() == null ? null : label.method().name());
|
||||
node.put("protected", label.isProtected());
|
||||
}
|
||||
|
||||
/** Cheap check so a non-PDF never pays for a parse attempt. */
|
||||
private static boolean looksLikePdf(byte[] content) {
|
||||
return content.length > 4
|
||||
&& content[0] == '%'
|
||||
&& content[1] == 'P'
|
||||
&& content[2] == 'D'
|
||||
&& content[3] == 'F';
|
||||
}
|
||||
|
||||
/**
|
||||
* A content hash is the field external systems most often key on - dedupe, chain-of-custody,
|
||||
* "have I already scanned this" - and they cannot compute it without the bytes we are sending.
|
||||
*/
|
||||
private static String sha256(byte[] content) {
|
||||
try {
|
||||
return HexFormat.of().formatHex(MessageDigest.getInstance("SHA-256").digest(content));
|
||||
} catch (NoSuchAlgorithmException e) {
|
||||
throw new IllegalStateException("SHA-256 is required by the Java platform", e);
|
||||
}
|
||||
}
|
||||
|
||||
private static String toIso(Calendar calendar) {
|
||||
return calendar == null ? null : calendar.toInstant().toString();
|
||||
}
|
||||
|
||||
private static String extensionOf(String filename) {
|
||||
if (filename == null) {
|
||||
return null;
|
||||
}
|
||||
int dot = filename.lastIndexOf('.');
|
||||
return dot < 0 || dot == filename.length() - 1
|
||||
? null
|
||||
: filename.substring(dot + 1).toLowerCase(Locale.ROOT);
|
||||
}
|
||||
}
|
||||
+552
@@ -0,0 +1,552 @@
|
||||
package stirling.software.proprietary.integration.api;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.util.Base64;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.List;
|
||||
import java.util.Locale;
|
||||
import java.util.Map;
|
||||
|
||||
import org.springframework.core.io.ByteArrayResource;
|
||||
import org.springframework.core.io.Resource;
|
||||
import org.springframework.http.HttpHeaders;
|
||||
import org.springframework.http.MediaType;
|
||||
import org.springframework.http.ResponseEntity;
|
||||
import org.springframework.web.bind.annotation.PostMapping;
|
||||
import org.springframework.web.bind.annotation.RequestHeader;
|
||||
import org.springframework.web.bind.annotation.RequestMapping;
|
||||
import org.springframework.web.bind.annotation.RequestParam;
|
||||
import org.springframework.web.bind.annotation.RestController;
|
||||
import org.springframework.web.multipart.MultipartFile;
|
||||
|
||||
import io.github.pixee.security.Filenames;
|
||||
import io.swagger.v3.oas.annotations.Operation;
|
||||
import io.swagger.v3.oas.annotations.tags.Tag;
|
||||
|
||||
import lombok.RequiredArgsConstructor;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
import stirling.software.common.model.ApplicationProperties;
|
||||
import stirling.software.common.service.AutomationRunContext;
|
||||
import stirling.software.common.service.InternalApiClient;
|
||||
import stirling.software.common.util.TempFileManager;
|
||||
import stirling.software.proprietary.service.AiToolResponseHeaders;
|
||||
|
||||
import tools.jackson.core.type.TypeReference;
|
||||
import tools.jackson.databind.JsonNode;
|
||||
import tools.jackson.databind.ObjectMapper;
|
||||
import tools.jackson.databind.node.ObjectNode;
|
||||
|
||||
/**
|
||||
* Posts the document flowing through a policy to a third-party HTTP API and folds the answer back
|
||||
* into the run.
|
||||
*
|
||||
* <p>This is the generic integration step: rather than a bespoke connector per vendor, an operator
|
||||
* defines an {@code API} connection (base URL + credentials) once and any policy can call a path
|
||||
* under it. The connection owns the host and the credentials; the step owns only the path and the
|
||||
* form fields, so a policy author can never aim the call somewhere else or read the secret.
|
||||
*
|
||||
* <p>Response handling is explicit rather than inferred, because the two useful behaviours destroy
|
||||
* different things when guessed wrong:
|
||||
*
|
||||
* <ul>
|
||||
* <li>{@code report} (default) - the document continues untouched and the API's answer rides
|
||||
* along in {@link AiToolResponseHeaders#TOOL_REPORT}. For call-outs that inspect or notify. A
|
||||
* {@code requireTrue} field turns the answer into a gate: the named JSON verdict must be true
|
||||
* or the step fails, so a scanner's "not clean" actually stops the run.
|
||||
* <li>{@code replace} - the response body <em>becomes</em> the document. For call-outs that
|
||||
* transform. Fails loudly if the API returns JSON or an empty body, instead of silently
|
||||
* dropping the document from the pipeline.
|
||||
* </ul>
|
||||
*/
|
||||
@Slf4j
|
||||
@RestController
|
||||
@RequestMapping("/api/v1/integration")
|
||||
@RequiredArgsConstructor
|
||||
@Tag(name = "Integrations", description = "Third-party integration steps.")
|
||||
public class ExternalApiCallController {
|
||||
|
||||
static final String MODE_REPORT = "report";
|
||||
static final String MODE_REPLACE = "replace";
|
||||
|
||||
/**
|
||||
* The report travels as an HTTP header, and Jetty caps a response header at 8KB by default. A
|
||||
* body larger than this is summarised rather than risking a header the container refuses to
|
||||
* write - which would fail the whole step over a merely verbose API.
|
||||
*/
|
||||
static final int MAX_REPORT_BODY_CHARS = 4096;
|
||||
|
||||
static final String BODY_MULTIPART = "multipart";
|
||||
static final String BODY_JSON = "json";
|
||||
static final String BODY_BINARY = "binary";
|
||||
|
||||
/** Field (multipart) and property (json) the auto-populated context is offered under. */
|
||||
static final String CONTEXT_FIELD = "stirlingContext";
|
||||
|
||||
private final ApiConnectionResolver connectionResolver;
|
||||
private final ExternalApiCaller caller;
|
||||
private final ObjectMapper objectMapper;
|
||||
private final TempFileManager tempFileManager;
|
||||
private final ApplicationProperties applicationProperties;
|
||||
|
||||
@PostMapping(value = "/external-api-call", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
|
||||
@Operation(
|
||||
summary = "Send the document to an external API",
|
||||
description =
|
||||
"Sends the document to a path under a stored API connection's base URL and"
|
||||
+ " either records the response as a step report or replaces the"
|
||||
+ " document with it. Fields, path and headers may reference"
|
||||
+ " {{document.*}}, {{classification.*}}, {{sensitivityLabel.*}} and"
|
||||
+ " {{run.*}}. Type:SISO")
|
||||
public ResponseEntity<Resource> call(
|
||||
@RequestParam("fileInput") MultipartFile fileInput,
|
||||
@RequestParam("connectionId") String connectionId,
|
||||
@RequestParam(value = "path", required = false) String path,
|
||||
@RequestParam(value = "method", defaultValue = "POST") String method,
|
||||
@RequestParam(value = "bodyMode", defaultValue = BODY_MULTIPART) String bodyMode,
|
||||
@RequestParam(value = "fileFieldName", defaultValue = "file") String fileFieldName,
|
||||
@RequestParam(value = "responseMode", defaultValue = MODE_REPORT) String responseMode,
|
||||
@RequestParam(value = "resultUrlPath", required = false) String resultUrlPath,
|
||||
@RequestParam(value = "resultUrlHeader", required = false) String resultUrlHeader,
|
||||
@RequestParam(value = "responseSelect", required = false) String responseSelect,
|
||||
@RequestParam(value = "requireTrue", required = false) String requireTrue,
|
||||
@RequestParam(value = "fields", required = false) String fields,
|
||||
@RequestParam(value = "bodyTemplate", required = false) String bodyTemplate,
|
||||
@RequestParam(value = "headers", required = false) String headers,
|
||||
@RequestParam(value = "includeContext", defaultValue = "false") boolean includeContext,
|
||||
@RequestParam(value = "includeFile", defaultValue = "true") boolean includeFile,
|
||||
@RequestHeader(value = InternalApiClient.POLICY_NAME_HEADER, required = false)
|
||||
String policyName,
|
||||
@RequestHeader(value = AutomationRunContext.RUN_ID_HEADER, required = false)
|
||||
String runId)
|
||||
throws IOException {
|
||||
|
||||
String mode = normalise(responseMode, MODE_REPORT, MODE_REPORT, MODE_REPLACE);
|
||||
String body = normalise(bodyMode, BODY_MULTIPART, BODY_MULTIPART, BODY_JSON, BODY_BINARY);
|
||||
String verb = parseMethod(method);
|
||||
|
||||
Long id = ApiConnectionResolver.connectionId(connectionId);
|
||||
if (id == null) {
|
||||
throw new IllegalArgumentException("'connectionId' is required");
|
||||
}
|
||||
ApiConnectionSettings settings = connectionResolver.resolve(id);
|
||||
|
||||
String filename = safeFileName(fileInput.getOriginalFilename());
|
||||
String contentType =
|
||||
fileInput.getContentType() == null
|
||||
? MediaType.APPLICATION_OCTET_STREAM_VALUE
|
||||
: fileInput.getContentType();
|
||||
byte[] content = fileInput.getBytes();
|
||||
|
||||
ObjectNode context =
|
||||
DocumentContext.build(fileInput, content, policyName, runId, objectMapper);
|
||||
|
||||
ExternalApiCaller.Response response =
|
||||
caller.dispatch(
|
||||
settings,
|
||||
verb,
|
||||
Placeholders.resolve(path, context, Placeholders.Escaping.URL_PATH),
|
||||
buildBody(
|
||||
body,
|
||||
bodyTemplate,
|
||||
includeFile,
|
||||
includeContext,
|
||||
context,
|
||||
fileFieldName,
|
||||
filename,
|
||||
contentType,
|
||||
content,
|
||||
resolveAll(parseJsonObject(fields, "fields"), context)),
|
||||
validatedHeaders(resolveAll(parseJsonObject(headers, "headers"), context)));
|
||||
|
||||
if (!response.isSuccess()) {
|
||||
// Fail the step: a policy that silently continued past a rejected call-out would
|
||||
// deliver documents the external system believes it never approved.
|
||||
throw new IOException(
|
||||
"External API returned HTTP " + response.status() + summarise(response));
|
||||
}
|
||||
|
||||
enforceVerdict(response, requireTrue);
|
||||
|
||||
return MODE_REPLACE.equals(mode)
|
||||
? replaceDocument(
|
||||
settings,
|
||||
response,
|
||||
filename,
|
||||
resultUrlPath,
|
||||
resultUrlHeader,
|
||||
responseSelect)
|
||||
: reportOnly(fileInput, filename, contentType, response);
|
||||
}
|
||||
|
||||
/**
|
||||
* Assemble the outbound body.
|
||||
*
|
||||
* <ul>
|
||||
* <li>{@code multipart} - the file plus form fields, what most upload APIs expect.
|
||||
* <li>{@code json} - a JSON object of the fields, with the context merged in and the file
|
||||
* base64'd under {@code content}. For APIs that take a document as JSON, and for
|
||||
* notify-style call-outs (with {@code includeFile=false}) that want the facts only.
|
||||
* <li>{@code binary} - the raw bytes as the body. For APIs that want the file and nothing
|
||||
* else; fields would have nowhere to go, so they are refused rather than dropped.
|
||||
* </ul>
|
||||
*/
|
||||
private ExternalApiCaller.Body buildBody(
|
||||
String bodyMode,
|
||||
String bodyTemplate,
|
||||
boolean includeFile,
|
||||
boolean includeContext,
|
||||
ObjectNode context,
|
||||
String fileFieldName,
|
||||
String filename,
|
||||
String contentType,
|
||||
byte[] content,
|
||||
Map<String, String> fields)
|
||||
throws IOException {
|
||||
|
||||
if (bodyTemplate != null && !bodyTemplate.isBlank()) {
|
||||
return templatedBody(bodyTemplate, context, filename, contentType, content);
|
||||
}
|
||||
switch (bodyMode) {
|
||||
case BODY_BINARY -> {
|
||||
if (!fields.isEmpty()) {
|
||||
throw new IllegalArgumentException(
|
||||
"bodyMode 'binary' sends only the document, so 'fields' cannot be"
|
||||
+ " sent; use 'headers' instead, or bodyMode 'multipart'.");
|
||||
}
|
||||
if (!includeFile) {
|
||||
throw new IllegalArgumentException(
|
||||
"bodyMode 'binary' with includeFile=false would send an empty body");
|
||||
}
|
||||
return ExternalApiCaller.raw(contentType, content);
|
||||
}
|
||||
case BODY_JSON -> {
|
||||
ObjectNode json = objectMapper.createObjectNode();
|
||||
fields.forEach(json::put);
|
||||
if (includeContext) {
|
||||
json.setAll(context);
|
||||
}
|
||||
if (includeFile) {
|
||||
json.put("filename", filename);
|
||||
json.put("contentType", contentType);
|
||||
json.put("content", Base64.getEncoder().encodeToString(content));
|
||||
}
|
||||
return ExternalApiCaller.raw(
|
||||
MediaType.APPLICATION_JSON_VALUE, objectMapper.writeValueAsBytes(json));
|
||||
}
|
||||
default -> {
|
||||
Map<String, String> all = new LinkedHashMap<>(fields);
|
||||
if (includeContext) {
|
||||
all.put(CONTEXT_FIELD, objectMapper.writeValueAsString(context));
|
||||
}
|
||||
if (!includeFile) {
|
||||
// Fields-only multipart: a notify-style call-out that wants the facts, not
|
||||
// the document.
|
||||
MultipartBody body = new MultipartBody();
|
||||
body.addFields(all);
|
||||
return new ExternalApiCaller.Body(body.contentType(), body.build());
|
||||
}
|
||||
return ExternalApiCaller.multipart(
|
||||
fileFieldName, filename, contentType, content, all);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* A caller-shaped JSON body: the template is resolved against the context, so an arbitrary
|
||||
* vendor payload can be expressed as config. {@code {{document.base64}}} carries the file
|
||||
* itself, which is how APIs that take a document nested inside a JSON document are reached.
|
||||
*
|
||||
* <p>The base64 is added to a copy of the context rather than the context proper: it is the
|
||||
* size of the file, and {@code stirlingContext} must not silently grow by a whole document.
|
||||
*/
|
||||
private ExternalApiCaller.Body templatedBody(
|
||||
String bodyTemplate,
|
||||
ObjectNode context,
|
||||
String filename,
|
||||
String contentType,
|
||||
byte[] content)
|
||||
throws IOException {
|
||||
JsonNode template;
|
||||
try {
|
||||
template = objectMapper.readTree(bodyTemplate);
|
||||
} catch (Exception e) {
|
||||
throw new IllegalArgumentException("api step 'bodyTemplate' must be valid JSON", e);
|
||||
}
|
||||
ObjectNode withFile = context.deepCopy();
|
||||
ObjectNode document = (ObjectNode) withFile.get("document");
|
||||
if (document != null) {
|
||||
document.put("base64", Base64.getEncoder().encodeToString(content));
|
||||
document.put("safeFilename", filename);
|
||||
document.put("resolvedContentType", contentType);
|
||||
}
|
||||
JsonNode resolved = Placeholders.resolveTree(template, withFile);
|
||||
return ExternalApiCaller.raw(
|
||||
MediaType.APPLICATION_JSON_VALUE, objectMapper.writeValueAsBytes(resolved));
|
||||
}
|
||||
|
||||
/** Resolve every value's placeholders against the context. */
|
||||
private Map<String, String> resolveAll(Map<String, String> values, ObjectNode context) {
|
||||
Map<String, String> out = new LinkedHashMap<>();
|
||||
values.forEach(
|
||||
(key, value) ->
|
||||
out.put(
|
||||
key,
|
||||
Placeholders.resolve(value, context, Placeholders.Escaping.NONE)));
|
||||
return out;
|
||||
}
|
||||
|
||||
/** Per-step headers, held to the same rules as a connection's static headers. */
|
||||
private Map<String, String> validatedHeaders(Map<String, String> headers) {
|
||||
headers.forEach(
|
||||
(name, value) -> {
|
||||
if (!ExternalApiHeaders.isValidName(name)) {
|
||||
throw new IllegalArgumentException(
|
||||
"api step 'headers' has an invalid header name: " + name);
|
||||
}
|
||||
if (ExternalApiHeaders.isReserved(name)) {
|
||||
throw new IllegalArgumentException(
|
||||
"api step 'headers' must not set '"
|
||||
+ name
|
||||
+ "'; it is set by the connection or the client");
|
||||
}
|
||||
if (!ExternalApiHeaders.isValidValue(value)) {
|
||||
// A resolved placeholder could carry a newline out of document metadata.
|
||||
throw new IllegalArgumentException(
|
||||
"api step 'headers' has an invalid value for '" + name + "'");
|
||||
}
|
||||
});
|
||||
return headers;
|
||||
}
|
||||
|
||||
private static String parseMethod(String method) {
|
||||
String verb = method == null ? "POST" : method.trim().toUpperCase(Locale.ROOT);
|
||||
// Only the verbs that carry a body; GET/DELETE would silently drop the document.
|
||||
if (!List.of("POST", "PUT", "PATCH").contains(verb)) {
|
||||
throw new IllegalArgumentException(
|
||||
"'method' must be POST, PUT or PATCH; got " + method);
|
||||
}
|
||||
return verb;
|
||||
}
|
||||
|
||||
private static String normalise(String value, String fallback, String... allowed) {
|
||||
String out =
|
||||
value == null || value.isBlank() ? fallback : value.trim().toLowerCase(Locale.ROOT);
|
||||
if (!List.of(allowed).contains(out)) {
|
||||
throw new IllegalArgumentException(
|
||||
"must be one of " + String.join(", ", allowed) + "; got " + value);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* Turn the response into the document that continues down the pipeline.
|
||||
*
|
||||
* <p>Three shapes of answer are accepted, because real APIs use all three: the document inline,
|
||||
* a URL to fetch it from, or an archive to pick it out of. Anything else fails the step rather
|
||||
* than putting a non-document into the pipeline for a later step to trip over.
|
||||
*/
|
||||
private ResponseEntity<Resource> replaceDocument(
|
||||
ApiConnectionSettings settings,
|
||||
ExternalApiCaller.Response response,
|
||||
String requestFilename,
|
||||
String resultUrlPath,
|
||||
String resultUrlHeader,
|
||||
String responseSelect)
|
||||
throws IOException {
|
||||
|
||||
ExternalApiCaller.Response payload = response;
|
||||
boolean followed = false;
|
||||
String url = resultUrl(response, resultUrlPath, resultUrlHeader);
|
||||
if (url != null) {
|
||||
// The URL came out of the response, so ResultUrls decides whether it may be fetched.
|
||||
payload =
|
||||
caller.getResult(
|
||||
settings, ResultUrls.validate(settings, url, applicationProperties));
|
||||
followed = true;
|
||||
if (!payload.isSuccess()) {
|
||||
throw new IOException(
|
||||
"Fetching the API's result URL returned HTTP " + payload.status());
|
||||
}
|
||||
}
|
||||
|
||||
if (payload.body().length == 0) {
|
||||
throw new IOException(
|
||||
"External API returned an empty body, so there is no document to replace with;"
|
||||
+ " use responseMode=report to keep the original.");
|
||||
}
|
||||
if (payload.isJson() && !followed) {
|
||||
throw new IOException(
|
||||
"External API returned JSON, which cannot replace the document. Use"
|
||||
+ " responseMode=report to keep the original and record the answer, or"
|
||||
+ " set resultUrlPath if the JSON points at the document.");
|
||||
}
|
||||
|
||||
String filename = ResultFiles.nameFor(payload, requestFilename);
|
||||
Resource result = ResultFiles.asResource(payload.body(), filename);
|
||||
|
||||
if (ResultFiles.isArchive(result)) {
|
||||
if (responseSelect == null || responseSelect.isBlank()) {
|
||||
// Handing a .zip to a step that expects a PDF fails later and more obscurely.
|
||||
throw new IOException(
|
||||
"External API returned an archive; set 'responseSelect' (e.g. *.pdf, or an"
|
||||
+ " index) to say which entry becomes the document.");
|
||||
}
|
||||
result = ResultFiles.selectFromArchive(result, responseSelect, tempFileManager);
|
||||
filename = result.getFilename();
|
||||
} else if (responseSelect != null && !responseSelect.isBlank()) {
|
||||
throw new IOException(
|
||||
"'responseSelect' was set but the API returned a single file, not an archive");
|
||||
}
|
||||
|
||||
MediaType type =
|
||||
payload.contentType() == null || ResultFiles.isArchiveName(filename)
|
||||
? MediaType.APPLICATION_OCTET_STREAM
|
||||
: MediaType.parseMediaType(payload.contentType().split(";")[0].trim());
|
||||
return ResponseEntity.ok()
|
||||
.contentType(type)
|
||||
.header(
|
||||
HttpHeaders.CONTENT_DISPOSITION,
|
||||
"attachment; filename=\"" + filename + "\"")
|
||||
.body(result);
|
||||
}
|
||||
|
||||
/** The result URL the API pointed at, from the body or a header; null when neither is set. */
|
||||
private String resultUrl(
|
||||
ExternalApiCaller.Response response, String resultUrlPath, String resultUrlHeader) {
|
||||
if (resultUrlHeader != null && !resultUrlHeader.isBlank()) {
|
||||
String value = response.header(resultUrlHeader.trim());
|
||||
if (value == null || value.isBlank()) {
|
||||
throw new IllegalArgumentException(
|
||||
"'resultUrlHeader' names '"
|
||||
+ resultUrlHeader
|
||||
+ "' but the response had no such header");
|
||||
}
|
||||
return value;
|
||||
}
|
||||
if (resultUrlPath == null || resultUrlPath.isBlank()) {
|
||||
return null;
|
||||
}
|
||||
JsonNode node = response.bodyAsJson(objectMapper);
|
||||
for (String segment : resultUrlPath.trim().split("\\.")) {
|
||||
if (node == null) {
|
||||
break;
|
||||
}
|
||||
node = node.get(segment);
|
||||
}
|
||||
if (node == null || !node.isValueNode() || node.asString().isBlank()) {
|
||||
throw new IllegalArgumentException(
|
||||
"'resultUrlPath' found no URL at '" + resultUrlPath + "' in the response");
|
||||
}
|
||||
return node.asString();
|
||||
}
|
||||
|
||||
/**
|
||||
* Gate the run on a boolean verdict in the API's JSON answer (e.g. Cloudmersive's {@code
|
||||
* CleanResult}). When {@code requireTrue} names a field - dotted for a nested one - that field
|
||||
* must be JSON {@code true}, or the step fails so the document is parked rather than delivered.
|
||||
* Fail-closed: a missing field, a non-boolean, a false, or a non-JSON body all stop the run.
|
||||
* This is what makes a scanner's "not clean" actually stop the pipeline.
|
||||
*/
|
||||
private void enforceVerdict(ExternalApiCaller.Response response, String requireTrue)
|
||||
throws IOException {
|
||||
if (requireTrue == null || requireTrue.isBlank()) {
|
||||
return;
|
||||
}
|
||||
JsonNode node = response.isJson() ? response.bodyAsJson(objectMapper) : null;
|
||||
for (String segment : requireTrue.trim().split("\\.")) {
|
||||
if (node == null) {
|
||||
break;
|
||||
}
|
||||
node = node.get(segment);
|
||||
}
|
||||
if (node == null || !node.asBoolean(false)) {
|
||||
throw new IOException(
|
||||
"External API verdict '"
|
||||
+ requireTrue.trim()
|
||||
+ "' was not true"
|
||||
+ summarise(response)
|
||||
+ "; the document was not approved, so the run was stopped.");
|
||||
}
|
||||
}
|
||||
|
||||
/** The document passes through; the API's answer rides in the report header. */
|
||||
private ResponseEntity<Resource> reportOnly(
|
||||
MultipartFile fileInput,
|
||||
String filename,
|
||||
String contentType,
|
||||
ExternalApiCaller.Response response)
|
||||
throws IOException {
|
||||
return ResponseEntity.ok()
|
||||
.contentType(MediaType.parseMediaType(contentType))
|
||||
.header(
|
||||
HttpHeaders.CONTENT_DISPOSITION,
|
||||
"attachment; filename=\"" + filename + "\"")
|
||||
.header(AiToolResponseHeaders.TOOL_REPORT, buildReport(response))
|
||||
.body(new ByteArrayResource(fileInput.getBytes()));
|
||||
}
|
||||
|
||||
/** A JSON object describing the call, small enough to survive as a header. */
|
||||
private String buildReport(ExternalApiCaller.Response response) {
|
||||
ObjectNode report = objectMapper.createObjectNode();
|
||||
report.put("status", response.status());
|
||||
report.put("contentType", response.contentType());
|
||||
if (response.isJson()) {
|
||||
try {
|
||||
JsonNode parsed = objectMapper.readTree(response.bodyAsText());
|
||||
String rendered = objectMapper.writeValueAsString(parsed);
|
||||
if (rendered.length() <= MAX_REPORT_BODY_CHARS) {
|
||||
report.set("body", parsed);
|
||||
} else {
|
||||
report.put("bodyTruncated", true);
|
||||
report.put("body", rendered.substring(0, MAX_REPORT_BODY_CHARS));
|
||||
}
|
||||
} catch (Exception e) {
|
||||
// Content-Type said JSON but the body is not; keep the step alive and say so.
|
||||
report.put("bodyParseError", e.getMessage());
|
||||
report.put("body", truncate(response.bodyAsText()));
|
||||
}
|
||||
} else {
|
||||
report.put("bodyBytes", response.body().length);
|
||||
}
|
||||
return objectMapper.writeValueAsString(report);
|
||||
}
|
||||
|
||||
/** A JSON object of string values, e.g. {@code {"policy":"strict"}}. */
|
||||
private Map<String, String> parseJsonObject(String json, String what) {
|
||||
if (json == null || json.isBlank()) {
|
||||
return Map.of();
|
||||
}
|
||||
Map<String, Object> raw;
|
||||
try {
|
||||
raw =
|
||||
objectMapper.readValue(
|
||||
json, new TypeReference<LinkedHashMap<String, Object>>() {});
|
||||
} catch (Exception e) {
|
||||
throw new IllegalArgumentException("api step '" + what + "' must be a JSON object", e);
|
||||
}
|
||||
Map<String, String> out = new LinkedHashMap<>();
|
||||
raw.forEach((key, value) -> out.put(key, value == null ? "" : value.toString()));
|
||||
return out;
|
||||
}
|
||||
|
||||
private String summarise(ExternalApiCaller.Response response) {
|
||||
String text = truncate(response.bodyAsText());
|
||||
return text.isBlank() ? "" : ": " + text;
|
||||
}
|
||||
|
||||
private static String truncate(String text) {
|
||||
if (text == null) {
|
||||
return "";
|
||||
}
|
||||
String oneLine = text.replaceAll("\\s+", " ").trim();
|
||||
return oneLine.length() <= MAX_REPORT_BODY_CHARS
|
||||
? oneLine
|
||||
: oneLine.substring(0, MAX_REPORT_BODY_CHARS) + "…";
|
||||
}
|
||||
|
||||
private static String safeFileName(String originalFilename) {
|
||||
String name = Filenames.toSimpleFileName(originalFilename);
|
||||
return (name == null || name.isBlank()) ? "document" : name;
|
||||
}
|
||||
}
|
||||
+299
@@ -0,0 +1,299 @@
|
||||
package stirling.software.proprietary.integration.api;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.net.URI;
|
||||
import java.net.http.HttpClient;
|
||||
import java.net.http.HttpRequest;
|
||||
import java.net.http.HttpResponse;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.time.Duration;
|
||||
import java.util.Base64;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.Map;
|
||||
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
import stirling.software.common.model.ApplicationProperties;
|
||||
|
||||
import tools.jackson.databind.JsonNode;
|
||||
import tools.jackson.databind.ObjectMapper;
|
||||
|
||||
/**
|
||||
* Performs the outbound call for an {@code API} connection.
|
||||
*
|
||||
* <p>Follows the established self-hosted outbound pattern (JDK {@link HttpClient}; see {@code
|
||||
* AccountLinkClient}): the client is injectable so tests can drive a real local server without
|
||||
* reaching the network.
|
||||
*/
|
||||
@Slf4j
|
||||
@Service
|
||||
public class ExternalApiCaller {
|
||||
|
||||
/**
|
||||
* Cap on a response we will read into memory. An external API returning something enormous is a
|
||||
* misconfiguration, and without a cap it would be a trivial way to OOM the server.
|
||||
*/
|
||||
static final int MAX_RESPONSE_BYTES = 64 * 1024 * 1024;
|
||||
|
||||
private static final Duration CONNECT_TIMEOUT = Duration.ofSeconds(10);
|
||||
|
||||
private final HttpClient httpClient;
|
||||
private final ApplicationProperties applicationProperties;
|
||||
private final ApiTokenCache tokenCache;
|
||||
|
||||
@Autowired
|
||||
public ExternalApiCaller(
|
||||
ApplicationProperties applicationProperties, ObjectMapper objectMapper) {
|
||||
this(
|
||||
HttpClient.newBuilder()
|
||||
.connectTimeout(CONNECT_TIMEOUT)
|
||||
// Following a redirect would re-target the request at a host the base URL
|
||||
// never authorised, undoing ExternalApiPaths. Let the caller see the 3xx.
|
||||
.followRedirects(HttpClient.Redirect.NEVER)
|
||||
.build(),
|
||||
applicationProperties,
|
||||
objectMapper);
|
||||
}
|
||||
|
||||
ExternalApiCaller(
|
||||
HttpClient httpClient,
|
||||
ApplicationProperties applicationProperties,
|
||||
ObjectMapper objectMapper) {
|
||||
this.httpClient = httpClient;
|
||||
this.applicationProperties = applicationProperties;
|
||||
this.tokenCache = new ApiTokenCache(httpClient, objectMapper);
|
||||
}
|
||||
|
||||
/** What the external API sent back, before the step decides what to do with it. */
|
||||
public record Response(
|
||||
int status, String contentType, byte[] body, Map<String, String> headers) {
|
||||
|
||||
public Response {
|
||||
headers = headers == null ? Map.of() : Map.copyOf(headers);
|
||||
}
|
||||
|
||||
/** A response header by name, case-insensitively; null when absent. */
|
||||
public String header(String name) {
|
||||
for (Map.Entry<String, String> entry : headers.entrySet()) {
|
||||
if (entry.getKey().equalsIgnoreCase(name)) {
|
||||
return entry.getValue();
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
JsonNode bodyAsJson(ObjectMapper objectMapper) {
|
||||
try {
|
||||
return objectMapper.readTree(bodyAsText());
|
||||
} catch (RuntimeException e) {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
public boolean isSuccess() {
|
||||
return status >= 200 && status < 300;
|
||||
}
|
||||
|
||||
public boolean isJson() {
|
||||
return contentType != null && contentType.toLowerCase().contains("json");
|
||||
}
|
||||
|
||||
public String bodyAsText() {
|
||||
return new String(body, StandardCharsets.UTF_8);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* POST a document to {@code path} under the connection's base URL as multipart/form-data.
|
||||
*
|
||||
* @throws IOException on transport failure or an oversized response
|
||||
*/
|
||||
public Response postFile(
|
||||
ApiConnectionSettings settings,
|
||||
String path,
|
||||
String fileFieldName,
|
||||
String filename,
|
||||
String fileContentType,
|
||||
byte[] content,
|
||||
Map<String, String> fields)
|
||||
throws IOException {
|
||||
return dispatch(
|
||||
settings,
|
||||
"POST",
|
||||
path,
|
||||
multipart(fileFieldName, filename, fileContentType, content, fields),
|
||||
Map.of());
|
||||
}
|
||||
|
||||
/** A request body plus the Content-Type that describes it. */
|
||||
record Body(String contentType, HttpRequest.BodyPublisher publisher) {}
|
||||
|
||||
static Body multipart(
|
||||
String fileFieldName,
|
||||
String filename,
|
||||
String fileContentType,
|
||||
byte[] content,
|
||||
Map<String, String> fields)
|
||||
throws IOException {
|
||||
MultipartBody body = new MultipartBody();
|
||||
body.addFields(fields);
|
||||
body.addFile(fileFieldName, filename, fileContentType, content);
|
||||
return new Body(body.contentType(), body.build());
|
||||
}
|
||||
|
||||
/** A body of caller-built bytes, e.g. a JSON document or the raw file. */
|
||||
static Body raw(String contentType, byte[] content) {
|
||||
return new Body(contentType, HttpRequest.BodyPublishers.ofByteArray(content));
|
||||
}
|
||||
|
||||
/**
|
||||
* Send {@code body} to {@code path} under the connection's base URL.
|
||||
*
|
||||
* @param method POST, PUT or PATCH - the verbs that carry a body
|
||||
* @param extraHeaders per-step headers, already validated by the caller
|
||||
*/
|
||||
public Response dispatch(
|
||||
ApiConnectionSettings settings,
|
||||
String method,
|
||||
String path,
|
||||
Body body,
|
||||
Map<String, String> extraHeaders)
|
||||
throws IOException {
|
||||
|
||||
URI target = ExternalApiPaths.resolve(settings.baseUri(), path);
|
||||
// Re-check at dispatch: save-time validation cannot see a DNS record re-pointed at a
|
||||
// private address afterwards.
|
||||
ApiIntegrationValidator.requirePublicHost(
|
||||
settings, applicationProperties, "API connection base URL");
|
||||
|
||||
Response response = attempt(settings, method, target, body, extraHeaders);
|
||||
if (response.status() == 401 && settings.authType() == ApiAuthType.TOKEN_LOGIN) {
|
||||
// The cached token was rejected - expired early, or revoked. One fresh login and
|
||||
// one retry; if that also 401s the credentials are wrong and the step says so.
|
||||
log.debug("[external-api] token rejected by {}; re-authenticating", target.getHost());
|
||||
tokenCache.invalidate(settings);
|
||||
response = attempt(settings, method, target, body, extraHeaders);
|
||||
}
|
||||
return response;
|
||||
}
|
||||
|
||||
private Response attempt(
|
||||
ApiConnectionSettings settings,
|
||||
String method,
|
||||
URI target,
|
||||
Body body,
|
||||
Map<String, String> extraHeaders)
|
||||
throws IOException {
|
||||
HttpRequest.Builder request =
|
||||
HttpRequest.newBuilder(target)
|
||||
.timeout(Duration.ofSeconds(settings.timeoutSeconds()))
|
||||
.header("Content-Type", body.contentType())
|
||||
.method(method, body.publisher());
|
||||
applyHeaders(request, settings);
|
||||
// Step headers last so a step can override a connection default, but never the auth
|
||||
// header: ExternalApiHeaders rejects reserved names before we get here.
|
||||
extraHeaders.forEach(request::header);
|
||||
return send(httpClient, request.build(), target);
|
||||
}
|
||||
|
||||
/**
|
||||
* GET an absolute result URL the API pointed us at.
|
||||
*
|
||||
* <p>Takes a {@link URI} rather than a string so it cannot be called with something unchecked:
|
||||
* the only way to obtain one is {@link ResultUrls#validate}, which is where the host allowlist
|
||||
* lives. Credentials are deliberately not sent - the URL is usually a presigned link on another
|
||||
* host, and forwarding the connection's token there would leak it to a third party.
|
||||
*/
|
||||
public Response getResult(ApiConnectionSettings settings, URI target) throws IOException {
|
||||
HttpRequest request =
|
||||
HttpRequest.newBuilder(target)
|
||||
.timeout(Duration.ofSeconds(settings.timeoutSeconds()))
|
||||
.GET()
|
||||
.build();
|
||||
return send(httpClient, request, target);
|
||||
}
|
||||
|
||||
/** GET {@code path} under the connection's base URL. */
|
||||
public Response get(ApiConnectionSettings settings, String path) throws IOException {
|
||||
URI target = ExternalApiPaths.resolve(settings.baseUri(), path);
|
||||
ApiIntegrationValidator.requirePublicHost(
|
||||
settings, applicationProperties, "API connection base URL");
|
||||
|
||||
HttpRequest.Builder request =
|
||||
HttpRequest.newBuilder(target)
|
||||
.timeout(Duration.ofSeconds(settings.timeoutSeconds()))
|
||||
.GET();
|
||||
applyHeaders(request, settings);
|
||||
return send(httpClient, request.build(), target);
|
||||
}
|
||||
|
||||
static Response send(HttpClient httpClient, HttpRequest request, URI target)
|
||||
throws IOException {
|
||||
HttpResponse<byte[]> response;
|
||||
try {
|
||||
response = httpClient.send(request, HttpResponse.BodyHandlers.ofByteArray());
|
||||
} catch (InterruptedException e) {
|
||||
Thread.currentThread().interrupt();
|
||||
throw new IOException("Interrupted calling " + safeTarget(target), e);
|
||||
} catch (IOException e) {
|
||||
// The message can carry the host but never the credentials, which live in headers.
|
||||
throw new IOException(
|
||||
"Failed to call " + safeTarget(target) + ": " + e.getMessage(), e);
|
||||
}
|
||||
byte[] body = response.body() == null ? new byte[0] : response.body();
|
||||
if (body.length > MAX_RESPONSE_BYTES) {
|
||||
throw new IOException(
|
||||
"Response from "
|
||||
+ safeTarget(target)
|
||||
+ " exceeds the "
|
||||
+ MAX_RESPONSE_BYTES
|
||||
+ " byte limit");
|
||||
}
|
||||
String contentType = response.headers().firstValue("content-type").orElse(null);
|
||||
Map<String, String> headers = new LinkedHashMap<>();
|
||||
response.headers()
|
||||
.map()
|
||||
.forEach((name, values) -> headers.put(name, String.join(", ", values)));
|
||||
log.debug("[external-api] {} -> HTTP {}", safeTarget(target), response.statusCode());
|
||||
return new Response(response.statusCode(), contentType, body, headers);
|
||||
}
|
||||
|
||||
private void applyHeaders(HttpRequest.Builder request, ApiConnectionSettings settings)
|
||||
throws IOException {
|
||||
settings.headers().forEach(request::header);
|
||||
switch (settings.authType()) {
|
||||
case BEARER -> request.header("Authorization", "Bearer " + settings.token());
|
||||
case HEADER ->
|
||||
request.header(
|
||||
settings.headerName(),
|
||||
settings.headerPrefix() == null
|
||||
? settings.token()
|
||||
: settings.headerPrefix() + " " + settings.token());
|
||||
case BASIC ->
|
||||
request.header(
|
||||
"Authorization",
|
||||
"Basic "
|
||||
+ Base64.getEncoder()
|
||||
.encodeToString(
|
||||
(settings.username()
|
||||
+ ":"
|
||||
+ settings.password())
|
||||
.getBytes(StandardCharsets.UTF_8)));
|
||||
case TOKEN_LOGIN -> {
|
||||
Map.Entry<String, String> auth = tokenCache.authHeader(settings);
|
||||
request.header(auth.getKey(), auth.getValue());
|
||||
}
|
||||
case NONE -> {
|
||||
/* no credentials */
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** Scheme, host and path only: a query string could carry a token an operator put there. */
|
||||
private static String safeTarget(URI target) {
|
||||
return target.getScheme() + "://" + target.getAuthority() + target.getPath();
|
||||
}
|
||||
}
|
||||
+72
@@ -0,0 +1,72 @@
|
||||
package stirling.software.proprietary.integration.api;
|
||||
|
||||
import java.util.Locale;
|
||||
import java.util.Set;
|
||||
|
||||
/**
|
||||
* Validation for operator-supplied HTTP header names and values.
|
||||
*
|
||||
* <p>Header values reach the wire verbatim, so a value carrying CR/LF could splice extra headers -
|
||||
* or a whole second request - into the stream. Names and values are therefore checked against the
|
||||
* RFC 7230 grammar rather than trusted.
|
||||
*/
|
||||
public final class ExternalApiHeaders {
|
||||
|
||||
/**
|
||||
* Headers a connection may not set as a static header. Authentication has exactly one path
|
||||
* ({@code authType} + {@code token}) so credentials cannot be smuggled in as a "static" header
|
||||
* that bypasses the auth validation; the rest are framing headers owned by the HTTP client,
|
||||
* where a caller-set value would contradict the body actually sent.
|
||||
*/
|
||||
private static final Set<String> RESERVED =
|
||||
Set.of(
|
||||
"authorization",
|
||||
"proxy-authorization",
|
||||
"host",
|
||||
"content-length",
|
||||
"transfer-encoding",
|
||||
"connection",
|
||||
"upgrade",
|
||||
"expect");
|
||||
|
||||
private ExternalApiHeaders() {}
|
||||
|
||||
/** RFC 7230 {@code token}: the only characters legal in a header name. */
|
||||
public static boolean isValidName(String name) {
|
||||
if (name == null || name.isEmpty()) {
|
||||
return false;
|
||||
}
|
||||
for (int i = 0; i < name.length(); i++) {
|
||||
if (!isTokenChar(name.charAt(i))) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
/** Visible ASCII, space and horizontal tab. Excludes CR/LF and NUL, which would inject. */
|
||||
public static boolean isValidValue(String value) {
|
||||
if (value == null) {
|
||||
return false;
|
||||
}
|
||||
for (int i = 0; i < value.length(); i++) {
|
||||
char c = value.charAt(i);
|
||||
boolean printable = c >= 0x20 && c <= 0x7E;
|
||||
if (!printable && c != '\t') {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
public static boolean isReserved(String name) {
|
||||
return name != null && RESERVED.contains(name.toLowerCase(Locale.ROOT));
|
||||
}
|
||||
|
||||
private static boolean isTokenChar(char c) {
|
||||
return (c >= 'a' && c <= 'z')
|
||||
|| (c >= 'A' && c <= 'Z')
|
||||
|| (c >= '0' && c <= '9')
|
||||
|| "!#$%&'*+-.^_`|~".indexOf(c) >= 0;
|
||||
}
|
||||
}
|
||||
+120
@@ -0,0 +1,120 @@
|
||||
package stirling.software.proprietary.integration.api;
|
||||
|
||||
import java.net.URI;
|
||||
import java.net.URISyntaxException;
|
||||
import java.util.Locale;
|
||||
|
||||
/**
|
||||
* Resolves a step-supplied relative path under a connection's operator-set base URL.
|
||||
*
|
||||
* <p>This is the control that keeps the external-API step from being an SSRF primitive. The base
|
||||
* URL comes from an {@code IntegrationConfig} only someone with manage rights can edit; the path
|
||||
* comes from a pipeline step, which is a far weaker trust boundary. Everything here exists to
|
||||
* guarantee that a path can address a resource <em>under</em> the base and nothing else.
|
||||
*
|
||||
* <p>{@link URI#resolve} is deliberately not used: resolving the protocol-relative {@code
|
||||
* //evil.example} against {@code https://api.example.com/v1} yields {@code https://evil.example},
|
||||
* silently changing host. Instead the path is screened, appended textually, normalised, and then
|
||||
* the result is re-checked against the base - so a miss in the screen is still caught by the check.
|
||||
*/
|
||||
public final class ExternalApiPaths {
|
||||
|
||||
private ExternalApiPaths() {}
|
||||
|
||||
/**
|
||||
* @param base the connection's base URL, already validated as http(s) with a host
|
||||
* @param path a relative path, optionally with a query string; blank means the base itself
|
||||
* @throws IllegalArgumentException if the path is absolute, escapes the base, or carries
|
||||
* characters that could split the request line
|
||||
*/
|
||||
public static URI resolve(URI base, String path) {
|
||||
if (path == null || path.isBlank()) {
|
||||
return base;
|
||||
}
|
||||
String candidate = path.trim();
|
||||
screen(candidate);
|
||||
|
||||
if (!candidate.startsWith("/")) {
|
||||
candidate = "/" + candidate;
|
||||
}
|
||||
|
||||
URI resolved;
|
||||
try {
|
||||
resolved = new URI(base + candidate).normalize();
|
||||
} catch (URISyntaxException e) {
|
||||
throw new IllegalArgumentException(
|
||||
"api step 'path' is not a valid URL path: " + path, e);
|
||||
}
|
||||
requireSameOrigin(base, resolved, path);
|
||||
requireUnderBasePath(base, resolved, path);
|
||||
return resolved;
|
||||
}
|
||||
|
||||
/** Reject the shapes that could retarget the request before it is even assembled. */
|
||||
private static void screen(String path) {
|
||||
if (path.contains("://") || path.startsWith("//")) {
|
||||
throw new IllegalArgumentException(
|
||||
"api step 'path' must be relative to the connection's base URL, not an"
|
||||
+ " absolute or protocol-relative URL: "
|
||||
+ path);
|
||||
}
|
||||
for (int i = 0; i < path.length(); i++) {
|
||||
char c = path.charAt(i);
|
||||
// Control characters and spaces can split the request line; a backslash is normalised
|
||||
// to '/' by some servers and would sidestep the traversal check below.
|
||||
if (c <= 0x20 || c == 0x7F || c == '\\') {
|
||||
throw new IllegalArgumentException(
|
||||
"api step 'path' contains an illegal character: " + path);
|
||||
}
|
||||
}
|
||||
if (path.indexOf('#') >= 0) {
|
||||
throw new IllegalArgumentException(
|
||||
"api step 'path' must not contain a fragment: " + path);
|
||||
}
|
||||
// Percent-encoded dots would survive the normalise() below and be decoded by the target, so
|
||||
// a traversal must not be smuggled past us in encoded form.
|
||||
//
|
||||
// Only dots are rejected. An encoded slash or backslash is legitimate: Placeholders encodes
|
||||
// substituted values, so a filename containing '/' arrives here as %2F, where it is data
|
||||
// inside one segment rather than structure. Rejecting those would refuse ordinary filenames
|
||||
// while doing nothing for traversal, which needs the dots.
|
||||
String lower = path.toLowerCase(Locale.ROOT);
|
||||
if (lower.contains("%2e")) {
|
||||
throw new IllegalArgumentException(
|
||||
"api step 'path' must not percent-encode dots: " + path);
|
||||
}
|
||||
}
|
||||
|
||||
private static void requireSameOrigin(URI base, URI resolved, String original) {
|
||||
boolean sameOrigin =
|
||||
equalsIgnoreCase(base.getScheme(), resolved.getScheme())
|
||||
&& equalsIgnoreCase(base.getHost(), resolved.getHost())
|
||||
&& base.getPort() == resolved.getPort()
|
||||
&& resolved.getUserInfo() == null;
|
||||
if (!sameOrigin) {
|
||||
throw new IllegalArgumentException(
|
||||
"api step 'path' would change the target host; it must stay under the"
|
||||
+ " connection's base URL: "
|
||||
+ original);
|
||||
}
|
||||
}
|
||||
|
||||
private static void requireUnderBasePath(URI base, URI resolved, String original) {
|
||||
String basePath = base.getPath() == null ? "" : base.getPath();
|
||||
String resolvedPath = resolved.getPath() == null ? "" : resolved.getPath();
|
||||
// The base URL has its trailing slash stripped at parse time, so a base path of "/v1"
|
||||
// must match "/v1" exactly or be followed by a separator - never "/v1betray".
|
||||
boolean under =
|
||||
basePath.isEmpty()
|
||||
|| resolvedPath.equals(basePath)
|
||||
|| resolvedPath.startsWith(basePath + "/");
|
||||
if (!under) {
|
||||
throw new IllegalArgumentException(
|
||||
"api step 'path' escapes the connection's base path: " + original);
|
||||
}
|
||||
}
|
||||
|
||||
private static boolean equalsIgnoreCase(String a, String b) {
|
||||
return a == null ? b == null : a.equalsIgnoreCase(b);
|
||||
}
|
||||
}
|
||||
+68
@@ -0,0 +1,68 @@
|
||||
package stirling.software.proprietary.integration.api;
|
||||
|
||||
import java.util.Map;
|
||||
|
||||
import org.springframework.stereotype.Component;
|
||||
|
||||
import lombok.RequiredArgsConstructor;
|
||||
|
||||
import stirling.software.proprietary.integration.model.IntegrationType;
|
||||
import stirling.software.proprietary.policy.engine.PipelineStepValidator;
|
||||
import stirling.software.proprietary.policy.model.PipelineStep;
|
||||
|
||||
/**
|
||||
* Authorization-checks the {@code connectionId} of any integration step, on the request thread.
|
||||
*
|
||||
* <p>This is what stops an integration step being a confused deputy. A step names a connection by
|
||||
* id, and the worker thread that runs it has no principal - so {@link ApiConnectionResolver} lets
|
||||
* the lookup through unchecked there, exactly as the S3 resolver does. Without this validator a
|
||||
* caller could put any id in a step and have the server dial that tenant's endpoint with that
|
||||
* tenant's stored credentials. Resolving here, while the caller is still on the thread, forces the
|
||||
* ownership check to run.
|
||||
*
|
||||
* <p>Registered as a {@link PipelineStepValidator} so both entry points cover it: save-time
|
||||
* validation of a stored policy, and {@code PolicyController}'s ad-hoc gate.
|
||||
*/
|
||||
@Component
|
||||
@RequiredArgsConstructor
|
||||
public class IntegrationStepValidator implements PipelineStepValidator {
|
||||
|
||||
static final String CONNECTION_ID_PARAM = "connectionId";
|
||||
private static final String INTEGRATION_PREFIX = "/api/v1/integration/";
|
||||
|
||||
/**
|
||||
* Which connection type each integration step dereferences. A step under {@link
|
||||
* #INTEGRATION_PREFIX} that is absent here is rejected rather than waved through, so a new
|
||||
* endpoint cannot quietly skip this check by forgetting to register.
|
||||
*/
|
||||
private static final Map<String, IntegrationType> STEP_CONNECTION_TYPES =
|
||||
Map.of(
|
||||
"/api/v1/integration/external-api-call", IntegrationType.API,
|
||||
"/api/v1/integration/purview-apply-label", IntegrationType.PURVIEW,
|
||||
"/api/v1/integration/purview-read-label", IntegrationType.PURVIEW,
|
||||
"/api/v1/integration/consigno-submit", IntegrationType.CONSIGNO,
|
||||
"/api/v1/integration/consigno-fetch-signed", IntegrationType.CONSIGNO);
|
||||
|
||||
private final ApiConnectionResolver connectionResolver;
|
||||
|
||||
@Override
|
||||
public void validate(PipelineStep step) {
|
||||
String operation = step.operation();
|
||||
if (operation == null || !operation.startsWith(INTEGRATION_PREFIX)) {
|
||||
return;
|
||||
}
|
||||
IntegrationType type = STEP_CONNECTION_TYPES.get(operation);
|
||||
if (type == null) {
|
||||
throw new IllegalArgumentException("unknown integration step: " + operation);
|
||||
}
|
||||
Long connectionId =
|
||||
ApiConnectionResolver.connectionId(step.parameters().get(CONNECTION_ID_PARAM));
|
||||
if (connectionId == null) {
|
||||
throw new IllegalArgumentException(
|
||||
operation + " requires a '" + CONNECTION_ID_PARAM + "' parameter");
|
||||
}
|
||||
// Throws if the connection is missing, the wrong type, disabled, or not usable by the
|
||||
// caller. The parsed settings are discarded: this call is the check.
|
||||
connectionResolver.resolveConfig(connectionId, type);
|
||||
}
|
||||
}
|
||||
+101
@@ -0,0 +1,101 @@
|
||||
package stirling.software.proprietary.integration.api;
|
||||
|
||||
import java.io.ByteArrayOutputStream;
|
||||
import java.io.IOException;
|
||||
import java.net.http.HttpRequest;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.security.SecureRandom;
|
||||
import java.util.Base64;
|
||||
import java.util.Map;
|
||||
|
||||
/**
|
||||
* Builds a {@code multipart/form-data} body for the JDK HTTP client, which has no multipart
|
||||
* publisher of its own.
|
||||
*
|
||||
* <p>The body is assembled in memory. Callers bound the document size before getting here; the
|
||||
* external-API step is for API-shaped payloads, not bulk transfer.
|
||||
*/
|
||||
final class MultipartBody {
|
||||
|
||||
private final String boundary;
|
||||
private final ByteArrayOutputStream out = new ByteArrayOutputStream();
|
||||
|
||||
MultipartBody() {
|
||||
byte[] random = new byte[16];
|
||||
new SecureRandom().nextBytes(random);
|
||||
this.boundary =
|
||||
"StirlingBoundary" + Base64.getUrlEncoder().withoutPadding().encodeToString(random);
|
||||
}
|
||||
|
||||
String contentType() {
|
||||
return "multipart/form-data; boundary=" + boundary;
|
||||
}
|
||||
|
||||
/**
|
||||
* @throws IllegalArgumentException if the <em>name</em> could break out of its part header;
|
||||
* names come from step parameters, so they are checked rather than trusted
|
||||
*/
|
||||
MultipartBody addField(String name, String value) throws IOException {
|
||||
requireSafe(name, "field name");
|
||||
writeAscii("--" + boundary + "\r\n");
|
||||
writeAscii("Content-Disposition: form-data; name=\"" + name + "\"\r\n\r\n");
|
||||
// The value is body, not header: quotes, newlines and backslashes are ordinary data here
|
||||
// and must survive untouched. Checking it like a header rejected every JSON value - which
|
||||
// is most of them, the auto-populated context included.
|
||||
out.write(value.getBytes(StandardCharsets.UTF_8));
|
||||
writeAscii("\r\n");
|
||||
return this;
|
||||
}
|
||||
|
||||
MultipartBody addFile(String name, String filename, String contentType, byte[] content)
|
||||
throws IOException {
|
||||
requireSafe(name, "file field name");
|
||||
requireSafe(filename, "filename");
|
||||
writeAscii("--" + boundary + "\r\n");
|
||||
writeAscii(
|
||||
"Content-Disposition: form-data; name=\""
|
||||
+ name
|
||||
+ "\"; filename=\""
|
||||
+ filename
|
||||
+ "\"\r\n");
|
||||
writeAscii("Content-Type: " + contentType + "\r\n\r\n");
|
||||
out.write(content);
|
||||
writeAscii("\r\n");
|
||||
return this;
|
||||
}
|
||||
|
||||
HttpRequest.BodyPublisher build() throws IOException {
|
||||
writeAscii("--" + boundary + "--\r\n");
|
||||
return HttpRequest.BodyPublishers.ofByteArray(out.toByteArray());
|
||||
}
|
||||
|
||||
MultipartBody addFields(Map<String, String> fields) throws IOException {
|
||||
for (Map.Entry<String, String> entry : fields.entrySet()) {
|
||||
addField(entry.getKey(), entry.getValue());
|
||||
}
|
||||
return this;
|
||||
}
|
||||
|
||||
/**
|
||||
* A quote, CR, LF or backslash in a <em>part header</em> - a field name or filename - would let
|
||||
* it close the quoted string and forge headers of its own. Values are not checked: they are
|
||||
* body, and the boundary that delimits them is 16 random bytes minted per request, so a value
|
||||
* cannot end its own part.
|
||||
*/
|
||||
private static void requireSafe(String value, String what) {
|
||||
if (value == null) {
|
||||
throw new IllegalArgumentException("api step " + what + " must not be null");
|
||||
}
|
||||
if (value.indexOf('"') >= 0
|
||||
|| value.indexOf('\r') >= 0
|
||||
|| value.indexOf('\n') >= 0
|
||||
|| value.indexOf('\\') >= 0) {
|
||||
throw new IllegalArgumentException(
|
||||
"api step " + what + " contains an illegal character: " + value);
|
||||
}
|
||||
}
|
||||
|
||||
private void writeAscii(String text) throws IOException {
|
||||
out.write(text.getBytes(StandardCharsets.UTF_8));
|
||||
}
|
||||
}
|
||||
+153
@@ -0,0 +1,153 @@
|
||||
package stirling.software.proprietary.integration.api;
|
||||
|
||||
import java.util.regex.Matcher;
|
||||
import java.util.regex.Pattern;
|
||||
|
||||
import tools.jackson.databind.JsonNode;
|
||||
import tools.jackson.databind.node.ArrayNode;
|
||||
import tools.jackson.databind.node.ObjectNode;
|
||||
import tools.jackson.databind.node.StringNode;
|
||||
|
||||
/**
|
||||
* Substitutes {@code {{dotted.path}}} references against the {@link DocumentContext}.
|
||||
*
|
||||
* <p>This is what lets one step satisfy APIs that disagree about payload shape. Rather than a
|
||||
* connector per vendor, an operator writes the field names the vendor expects and fills them from
|
||||
* context - {@code {"sha256": "{{document.sha256}}", "class": "{{sensitivityLabel.name}}"}}.
|
||||
*
|
||||
* <p>Deliberately not a template language: dotted lookup and nothing else. No expressions, no
|
||||
* control flow, no method calls - a step definition is lower-trust than server config, and the
|
||||
* whole point of a template engine (evaluating what it is given) is the thing to avoid here.
|
||||
*/
|
||||
final class Placeholders {
|
||||
|
||||
private static final Pattern PLACEHOLDER = Pattern.compile("\\{\\{\\s*([\\w.]+)\\s*}}");
|
||||
|
||||
/** How a resolved value is escaped for the position it lands in. */
|
||||
enum Escaping {
|
||||
/** Verbatim: form fields and header values, which are validated separately. */
|
||||
NONE,
|
||||
/** Percent-encoded: a path segment, where a stray slash would change the target. */
|
||||
URL_PATH
|
||||
}
|
||||
|
||||
private Placeholders() {}
|
||||
|
||||
/**
|
||||
* @param template text that may contain {@code {{...}}} references; null passes through
|
||||
* @param context the object to resolve against
|
||||
* @throws IllegalArgumentException if a reference names something the context does not hold, so
|
||||
* a typo surfaces as an error instead of silently sending an empty value
|
||||
*/
|
||||
static String resolve(String template, JsonNode context, Escaping escaping) {
|
||||
if (template == null || template.isEmpty()) {
|
||||
return template;
|
||||
}
|
||||
Matcher matcher = PLACEHOLDER.matcher(template);
|
||||
StringBuilder out = new StringBuilder();
|
||||
while (matcher.find()) {
|
||||
String path = matcher.group(1);
|
||||
JsonNode value = lookup(context, path);
|
||||
if (value == null || value.isMissingNode()) {
|
||||
throw new IllegalArgumentException(
|
||||
"unknown placeholder '{{"
|
||||
+ path
|
||||
+ "}}'; available: document.*, classification.*,"
|
||||
+ " sensitivityLabel.*, run.*");
|
||||
}
|
||||
matcher.appendReplacement(out, Matcher.quoteReplacement(render(value, escaping)));
|
||||
}
|
||||
matcher.appendTail(out);
|
||||
return out.toString();
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve every string in a JSON tree, in place, leaving structure and non-strings alone.
|
||||
*
|
||||
* <p>This is what lets one step post an arbitrary vendor-shaped body - a nested {@code
|
||||
* documents[0].data} as readily as a flat field - without a connector per vendor.
|
||||
*/
|
||||
static JsonNode resolveTree(JsonNode node, JsonNode context) {
|
||||
if (node instanceof ObjectNode object) {
|
||||
for (String name : new java.util.ArrayList<>(object.propertyNames())) {
|
||||
object.set(name, resolveTree(object.get(name), context));
|
||||
}
|
||||
return object;
|
||||
}
|
||||
if (node instanceof ArrayNode array) {
|
||||
for (int i = 0; i < array.size(); i++) {
|
||||
array.set(i, resolveTree(array.get(i), context));
|
||||
}
|
||||
return array;
|
||||
}
|
||||
if (node != null && node.isString()) {
|
||||
return StringNode.valueOf(resolve(node.asString(), context, Escaping.NONE));
|
||||
}
|
||||
return node;
|
||||
}
|
||||
|
||||
/** Whether the text references anything at all, so callers can skip resolving. */
|
||||
static boolean hasPlaceholder(String text) {
|
||||
return text != null && PLACEHOLDER.matcher(text).find();
|
||||
}
|
||||
|
||||
private static JsonNode lookup(JsonNode context, String path) {
|
||||
JsonNode node = context;
|
||||
for (String segment : path.split("\\.")) {
|
||||
if (node == null || !node.isObject()) {
|
||||
return null;
|
||||
}
|
||||
node = node.get(segment);
|
||||
}
|
||||
return node;
|
||||
}
|
||||
|
||||
/**
|
||||
* A null in context renders empty rather than the literal "null": absent metadata is a normal
|
||||
* state, and "null" in a vendor's field would be a value, not an absence.
|
||||
*/
|
||||
private static String render(JsonNode value, Escaping escaping) {
|
||||
String text;
|
||||
if (value.isNull()) {
|
||||
text = "";
|
||||
} else if (value.isValueNode()) {
|
||||
text = value.asString();
|
||||
} else {
|
||||
// An object or array (e.g. {{classification}}) renders as its JSON.
|
||||
text = value.toString();
|
||||
}
|
||||
return escaping == Escaping.URL_PATH ? urlEncodePathSegment(text) : text;
|
||||
}
|
||||
|
||||
/**
|
||||
* Encode for a path segment: a filename is the likeliest value to land in a path and may carry
|
||||
* a slash, which would otherwise read as structure rather than data.
|
||||
*
|
||||
* <p>Dots are left alone even though a traversal is made of them. Encoding them would be worse:
|
||||
* {@code %2E%2E} survives {@link java.net.URI#normalize()} and gets decoded by the target, so
|
||||
* the traversal would arrive intact and unexamined. Left raw, {@code ..} normalises here and is
|
||||
* caught by {@code ExternalApiPaths}' under-the-base check - the one place that can actually
|
||||
* see it.
|
||||
*/
|
||||
private static String urlEncodePathSegment(String text) {
|
||||
StringBuilder out = new StringBuilder(text.length());
|
||||
for (byte b : text.getBytes(java.nio.charset.StandardCharsets.UTF_8)) {
|
||||
char c = (char) (b & 0xFF);
|
||||
// RFC 3986 unreserved.
|
||||
boolean unreserved =
|
||||
(c >= 'a' && c <= 'z')
|
||||
|| (c >= 'A' && c <= 'Z')
|
||||
|| (c >= '0' && c <= '9')
|
||||
|| c == '-'
|
||||
|| c == '.'
|
||||
|| c == '_'
|
||||
|| c == '~';
|
||||
if (unreserved) {
|
||||
out.append(c);
|
||||
} else {
|
||||
out.append('%').append(String.format("%02X", b & 0xFF));
|
||||
}
|
||||
}
|
||||
return out.toString();
|
||||
}
|
||||
}
|
||||
+215
@@ -0,0 +1,215 @@
|
||||
package stirling.software.proprietary.integration.api;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Locale;
|
||||
import java.util.Map;
|
||||
|
||||
import org.springframework.core.io.ByteArrayResource;
|
||||
import org.springframework.core.io.Resource;
|
||||
|
||||
import stirling.software.common.util.TempFileManager;
|
||||
import stirling.software.common.util.ZipExtractionUtils;
|
||||
|
||||
/**
|
||||
* Works out which bytes, and under which name, a response should contribute to the pipeline.
|
||||
*
|
||||
* <p>Three things go wrong if this is left implicit:
|
||||
*
|
||||
* <ul>
|
||||
* <li><b>The name.</b> A step that replaces the document must name it for what came back, not for
|
||||
* what went out. Keeping the inbound name means a PDF-to-DOCX call-out yields a DOCX called
|
||||
* {@code .pdf}, and the next step's type check either waves it through or rejects it for the
|
||||
* wrong reason. The response's own {@code Content-Disposition} or {@code Content-Type} is the
|
||||
* only honest source.
|
||||
* <li><b>Archives.</b> Plenty of APIs answer with a ZIP even when one file was sent - ConsignO
|
||||
* returns "PDF (single) or ZIP (multiple)". Handing a {@code .zip} to a step expecting a PDF
|
||||
* is a confusing failure, so a step can select what it wanted out of the archive.
|
||||
* <li><b>Nothing useful at all.</b> An empty body or an error page is not a document, and saying
|
||||
* so beats letting it flow onward as one.
|
||||
* </ul>
|
||||
*/
|
||||
final class ResultFiles {
|
||||
|
||||
/** Extensions we can name from a content type; anything else keeps the server's filename. */
|
||||
private static final Map<String, String> EXTENSION_BY_TYPE =
|
||||
Map.ofEntries(
|
||||
Map.entry("application/pdf", "pdf"),
|
||||
Map.entry("application/zip", "zip"),
|
||||
Map.entry("application/json", "json"),
|
||||
Map.entry("text/plain", "txt"),
|
||||
Map.entry("text/html", "html"),
|
||||
Map.entry("image/png", "png"),
|
||||
Map.entry("image/jpeg", "jpg"),
|
||||
Map.entry("image/tiff", "tiff"),
|
||||
Map.entry("application/msword", "doc"),
|
||||
Map.entry(
|
||||
"application/vnd.openxmlformats-officedocument.wordprocessingml.document",
|
||||
"docx"),
|
||||
Map.entry("application/vnd.ms-excel", "xls"),
|
||||
Map.entry(
|
||||
"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
|
||||
"xlsx"));
|
||||
|
||||
private ResultFiles() {}
|
||||
|
||||
/**
|
||||
* The filename to give the returned bytes.
|
||||
*
|
||||
* <p>Prefers what the server said ({@code Content-Disposition}), then the base name of the
|
||||
* request with an extension derived from {@code Content-Type}, and only then the original name
|
||||
* unchanged.
|
||||
*/
|
||||
static String nameFor(ExternalApiCaller.Response response, String requestFilename) {
|
||||
String disposition = response.header("content-disposition");
|
||||
String fromServer = filenameFromDisposition(disposition);
|
||||
if (fromServer != null) {
|
||||
return fromServer;
|
||||
}
|
||||
String extension = extensionFor(response.contentType());
|
||||
if (extension == null) {
|
||||
return requestFilename;
|
||||
}
|
||||
return baseName(requestFilename) + "." + extension;
|
||||
}
|
||||
|
||||
/**
|
||||
* Pick the file a step asked for out of an archive.
|
||||
*
|
||||
* @param select a glob such as {@code *.pdf}, or a 0-based index such as {@code 1}
|
||||
* @throws IOException if nothing in the archive matches, naming what was there - a silent pick
|
||||
* of the wrong file would be worse than a failed step
|
||||
*/
|
||||
static Resource selectFromArchive(
|
||||
Resource archive, String select, TempFileManager tempFileManager) throws IOException {
|
||||
List<Resource> entries = ZipExtractionUtils.extractZip(archive, tempFileManager);
|
||||
if (entries.isEmpty()) {
|
||||
throw new IOException("The API returned an empty archive");
|
||||
}
|
||||
Integer index = asIndex(select);
|
||||
if (index != null) {
|
||||
if (index < 0 || index >= entries.size()) {
|
||||
throw new IOException(
|
||||
"'responseSelect' asked for entry "
|
||||
+ index
|
||||
+ " but the archive has "
|
||||
+ entries.size()
|
||||
+ ": "
|
||||
+ names(entries));
|
||||
}
|
||||
return entries.get(index);
|
||||
}
|
||||
List<Resource> matches = new ArrayList<>();
|
||||
for (Resource entry : entries) {
|
||||
if (matchesGlob(entry.getFilename(), select)) {
|
||||
matches.add(entry);
|
||||
}
|
||||
}
|
||||
if (matches.isEmpty()) {
|
||||
throw new IOException(
|
||||
"'responseSelect' matched nothing in the archive; it holds " + names(entries));
|
||||
}
|
||||
if (matches.size() > 1) {
|
||||
// Taking the first would be a coin toss the operator did not ask for.
|
||||
throw new IOException(
|
||||
"'responseSelect' matched "
|
||||
+ matches.size()
|
||||
+ " entries ("
|
||||
+ names(matches)
|
||||
+ "); narrow it, or use an index");
|
||||
}
|
||||
return matches.get(0);
|
||||
}
|
||||
|
||||
/** Whether the chosen name is itself an archive, so its content type is not the entry's. */
|
||||
static boolean isArchiveName(String filename) {
|
||||
return filename != null && filename.toLowerCase(Locale.ROOT).endsWith(".zip");
|
||||
}
|
||||
|
||||
static boolean isArchive(Resource resource) throws IOException {
|
||||
return ZipExtractionUtils.isZip(resource);
|
||||
}
|
||||
|
||||
static Resource asResource(byte[] content, String filename) {
|
||||
return new ByteArrayResource(content) {
|
||||
@Override
|
||||
public String getFilename() {
|
||||
return filename;
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
/** Only {@code *} is supported, and only against the entry's own name. */
|
||||
private static boolean matchesGlob(String filename, String glob) {
|
||||
if (filename == null) {
|
||||
return false;
|
||||
}
|
||||
String name = filename.toLowerCase(Locale.ROOT);
|
||||
String pattern = glob.trim().toLowerCase(Locale.ROOT);
|
||||
String regex =
|
||||
java.util.Arrays.stream(pattern.split("\\*", -1))
|
||||
.map(java.util.regex.Pattern::quote)
|
||||
.reduce((a, b) -> a + ".*" + b)
|
||||
.orElse("");
|
||||
return name.matches(regex);
|
||||
}
|
||||
|
||||
private static Integer asIndex(String select) {
|
||||
try {
|
||||
return Integer.valueOf(select.trim());
|
||||
} catch (NumberFormatException e) {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
private static String names(List<Resource> entries) {
|
||||
return entries.stream().map(Resource::getFilename).toList().toString();
|
||||
}
|
||||
|
||||
/** {@code attachment; filename="signed.pdf"} or its RFC 5987 {@code filename*} form. */
|
||||
private static String filenameFromDisposition(String disposition) {
|
||||
if (disposition == null) {
|
||||
return null;
|
||||
}
|
||||
for (String part : disposition.split(";")) {
|
||||
String token = part.trim();
|
||||
String value = null;
|
||||
if (token.regionMatches(true, 0, "filename=", 0, 9)) {
|
||||
value = token.substring(9).trim();
|
||||
} else if (token.regionMatches(true, 0, "filename*=", 0, 10)) {
|
||||
value = token.substring(10).trim();
|
||||
int tick = value.lastIndexOf('\'');
|
||||
if (tick >= 0) {
|
||||
value = value.substring(tick + 1);
|
||||
}
|
||||
}
|
||||
if (value == null) {
|
||||
continue;
|
||||
}
|
||||
if (value.length() >= 2 && value.startsWith("\"") && value.endsWith("\"")) {
|
||||
value = value.substring(1, value.length() - 1);
|
||||
}
|
||||
// The name comes from the remote server, so it is treated as data: strip any path it
|
||||
// tries to bring with it rather than letting it steer where anything is written.
|
||||
String simple = io.github.pixee.security.Filenames.toSimpleFileName(value);
|
||||
if (simple != null && !simple.isBlank()) {
|
||||
return simple;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
private static String extensionFor(String contentType) {
|
||||
if (contentType == null) {
|
||||
return null;
|
||||
}
|
||||
String type = contentType.split(";")[0].trim().toLowerCase(Locale.ROOT);
|
||||
return EXTENSION_BY_TYPE.get(type);
|
||||
}
|
||||
|
||||
private static String baseName(String filename) {
|
||||
int dot = filename.lastIndexOf('.');
|
||||
return dot <= 0 ? filename : filename.substring(0, dot);
|
||||
}
|
||||
}
|
||||
+108
@@ -0,0 +1,108 @@
|
||||
package stirling.software.proprietary.integration.api;
|
||||
|
||||
import java.net.URI;
|
||||
import java.net.URISyntaxException;
|
||||
import java.util.Locale;
|
||||
import java.util.Set;
|
||||
|
||||
import stirling.software.common.model.ApplicationProperties;
|
||||
import stirling.software.proprietary.cluster.s3.S3Clients;
|
||||
|
||||
/**
|
||||
* Validates a result URL an external API asked us to fetch.
|
||||
*
|
||||
* <p>This is the most dangerous input in the whole feature and deserves saying plainly: unlike a
|
||||
* step's {@code path}, which an operator wrote, this URL is <em>chosen by the remote service at run
|
||||
* time</em>. Fetching whatever it names would hand any integration - or anything that has
|
||||
* compromised, spoofed, or MITM'd one - a server-side GET of its choosing, i.e. the cloud metadata
|
||||
* service. {@link ExternalApiPaths} cannot help here: the whole point of a result URL is that it
|
||||
* usually lives on a different host (a CDN or presigned object store), so "must be under the base
|
||||
* URL" would reject the normal case.
|
||||
*
|
||||
* <p>The rule is therefore an <em>operator-declared</em> allowlist: a result may come from the
|
||||
* connection's own host, or from a host named in the connection's {@code resultUrlHosts}. The
|
||||
* decision of which hosts are legitimate stays with whoever configured the connection, and never
|
||||
* with the response.
|
||||
*/
|
||||
final class ResultUrls {
|
||||
|
||||
private ResultUrls() {}
|
||||
|
||||
/**
|
||||
* @param url exactly as the API returned it
|
||||
* @return the URL to fetch
|
||||
* @throws IllegalArgumentException if the response named a host the operator did not authorise
|
||||
*/
|
||||
static URI validate(
|
||||
ApiConnectionSettings settings,
|
||||
String url,
|
||||
ApplicationProperties applicationProperties) {
|
||||
URI uri;
|
||||
try {
|
||||
uri = new URI(url.trim());
|
||||
} catch (URISyntaxException e) {
|
||||
throw new IllegalArgumentException(
|
||||
"The API returned a result URL that is not a valid URL: " + url, e);
|
||||
}
|
||||
String scheme = uri.getScheme() == null ? "" : uri.getScheme().toLowerCase(Locale.ROOT);
|
||||
if (!"http".equals(scheme) && !"https".equals(scheme)) {
|
||||
// file:, gopher:, jar: and friends are how a URL fetch becomes a local file read.
|
||||
throw new IllegalArgumentException(
|
||||
"The API returned a result URL that is not http(s): " + url);
|
||||
}
|
||||
String host = uri.getHost();
|
||||
if (host == null || host.isBlank()) {
|
||||
throw new IllegalArgumentException(
|
||||
"The API returned a result URL with no host: " + url);
|
||||
}
|
||||
if (uri.getUserInfo() != null) {
|
||||
// Credentials in a URL are also the classic way to make a host look like another one.
|
||||
throw new IllegalArgumentException(
|
||||
"The API returned a result URL carrying credentials, which is not accepted");
|
||||
}
|
||||
|
||||
if (!isAllowedHost(settings, host)) {
|
||||
throw new IllegalArgumentException(
|
||||
"The API returned a result URL on '"
|
||||
+ host
|
||||
+ "', which this connection does not allow. Add it to the connection's"
|
||||
+ " 'resultUrlHosts' if results are meant to come from there.");
|
||||
}
|
||||
// Even an allowlisted name must not resolve somewhere internal: a hostile or compromised
|
||||
// DNS record for cdn.vendor.example pointing at 169.254.169.254 would otherwise be obeyed.
|
||||
try {
|
||||
S3Clients.validateEndpointHost(
|
||||
uri,
|
||||
applicationProperties.getPolicies().isAllowPrivateApiEndpoints(),
|
||||
"API result URL",
|
||||
"set policies.allowPrivateApiEndpoints=true to opt in (e.g. for an on-prem"
|
||||
+ " integration).");
|
||||
} catch (IllegalStateException e) {
|
||||
throw new IllegalArgumentException(e.getMessage(), e);
|
||||
}
|
||||
return uri;
|
||||
}
|
||||
|
||||
/**
|
||||
* The connection's own host is implicitly allowed; anything else must be declared.
|
||||
*
|
||||
* <p>Package-private so the matching rule can be tested without a DNS lookup: {@link #validate}
|
||||
* additionally resolves the host, which fails closed and so cannot run against example hosts.
|
||||
*/
|
||||
static boolean isAllowedHost(ApiConnectionSettings settings, String host) {
|
||||
String candidate = host.toLowerCase(Locale.ROOT);
|
||||
if (candidate.equalsIgnoreCase(settings.baseUri().getHost())) {
|
||||
return true;
|
||||
}
|
||||
Set<String> allowed = settings.resultUrlHosts();
|
||||
for (String entry : allowed) {
|
||||
String allowedHost = entry.toLowerCase(Locale.ROOT);
|
||||
// An exact host, or a subdomain of it. Not a bare suffix match: "evilvendor.com"
|
||||
// must not be admitted by an entry of "vendor.com".
|
||||
if (candidate.equals(allowedHost) || candidate.endsWith("." + allowedHost)) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
}
|
||||
+19
@@ -52,6 +52,25 @@ public class IntegrationConfigController {
|
||||
return ResponseEntity.ok(service.toResponse(service.create(request, user), user));
|
||||
}
|
||||
|
||||
/**
|
||||
* What this caller may set up, so the UI offers the vendor presets and the free-form "custom
|
||||
* API" option only to those who can actually use them. The answer is computed here rather than
|
||||
* inferred client-side: hiding a button is presentation, and the service still refuses the call
|
||||
* regardless of what the client believed.
|
||||
*/
|
||||
@GetMapping("/capabilities")
|
||||
public ResponseEntity<IntegrationCapabilitiesResponse> capabilities(
|
||||
@AuthenticationPrincipal User user) {
|
||||
requireUser(user);
|
||||
return ResponseEntity.ok(
|
||||
new IntegrationCapabilitiesResponse(service.canAuthorCustomApi(user)));
|
||||
}
|
||||
|
||||
/**
|
||||
* @param customApi whether the caller may author a free-form API integration
|
||||
*/
|
||||
public record IntegrationCapabilitiesResponse(boolean customApi) {}
|
||||
|
||||
@GetMapping("/{id}")
|
||||
public ResponseEntity<IntegrationConfigResponse> get(
|
||||
@PathVariable Long id, @AuthenticationPrincipal User user) {
|
||||
|
||||
+12
-1
@@ -44,10 +44,13 @@ public class CredentialEncryption {
|
||||
private static volatile SecretKey key;
|
||||
|
||||
private final String configuredKey;
|
||||
private final boolean clusterEnabled;
|
||||
|
||||
public CredentialEncryption(
|
||||
@Value("${stirling.security.credentialEncryptionKey:}") String configuredKey) {
|
||||
@Value("${stirling.security.credentialEncryptionKey:}") String configuredKey,
|
||||
@Value("${cluster.enabled:false}") boolean clusterEnabled) {
|
||||
this.configuredKey = configuredKey;
|
||||
this.clusterEnabled = clusterEnabled;
|
||||
}
|
||||
|
||||
@PostConstruct
|
||||
@@ -64,6 +67,14 @@ public class CredentialEncryption {
|
||||
if (configured != null && !configured.isBlank()) {
|
||||
return new SecretKeySpec(Base64.getDecoder().decode(configured.trim()), ALGORITHM);
|
||||
}
|
||||
// Cluster nodes must share this key, so fail fast rather than generate a node-local one.
|
||||
if (clusterEnabled) {
|
||||
throw new IllegalStateException(
|
||||
"cluster.enabled=true requires a shared credential encryption key. Set"
|
||||
+ " STIRLING_CREDENTIAL_ENCRYPTION_KEY (or"
|
||||
+ " stirling.security.credentialEncryptionKey) to the same value on every"
|
||||
+ " node.");
|
||||
}
|
||||
return loadOrCreateKeyFile();
|
||||
}
|
||||
|
||||
|
||||
+6
-1
@@ -4,5 +4,10 @@ package stirling.software.proprietary.integration.model;
|
||||
public enum IntegrationType {
|
||||
S3,
|
||||
MCP,
|
||||
API
|
||||
/** A generic outbound HTTP endpoint a pipeline step can post a document to. */
|
||||
API,
|
||||
/** Microsoft Purview Information Protection: sensitivity-label taxonomy via Graph. */
|
||||
PURVIEW,
|
||||
/** ConsignO Cloud (Notarius) e-signature and notarization. */
|
||||
CONSIGNO
|
||||
}
|
||||
|
||||
+323
@@ -0,0 +1,323 @@
|
||||
package stirling.software.proprietary.integration.purview;
|
||||
|
||||
import java.io.ByteArrayOutputStream;
|
||||
import java.io.IOException;
|
||||
import java.io.InputStream;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.util.ArrayList;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.LinkedHashSet;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Optional;
|
||||
import java.util.Set;
|
||||
import java.util.function.Predicate;
|
||||
import java.util.regex.Matcher;
|
||||
import java.util.regex.Pattern;
|
||||
|
||||
import org.apache.pdfbox.pdmodel.PDDocument;
|
||||
import org.apache.pdfbox.pdmodel.PDDocumentCatalog;
|
||||
import org.apache.pdfbox.pdmodel.PDDocumentInformation;
|
||||
import org.apache.pdfbox.pdmodel.common.PDMetadata;
|
||||
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
/**
|
||||
* Reads and writes Microsoft Purview sensitivity labels on a PDF.
|
||||
*
|
||||
* <p>Microsoft documents <em>what</em> a label is - the {@code MSIP_Label_<GUID>_<Attribute>}
|
||||
* key/value set - but not <em>where</em> it lives inside a PDF; that detail sits inside the MIP
|
||||
* SDK, which is C++/.NET only and has no Java binding. This class therefore treats the two places a
|
||||
* PDF can hold such pairs as equally valid:
|
||||
*
|
||||
* <ul>
|
||||
* <li>the Document Information dictionary, whose custom entries are literally a key/value map;
|
||||
* <li>the XMP packet, where the same keys appear as properties.
|
||||
* </ul>
|
||||
*
|
||||
* <p>Reading is deliberately tolerant - it scans both and takes whichever yields a label - so a
|
||||
* document labelled by Acrobat, the MIP client, or another vendor is still understood. Writing
|
||||
* populates both, because a downstream reader may only look at one.
|
||||
*
|
||||
* <p>Scope: this applies the label <em>metadata</em>. It does not encrypt, and cannot: protection
|
||||
* is enforced by the Azure Rights Management service through the MIP SDK. A label whose policy
|
||||
* demands encryption will be marked here but not protected, which {@link #apply} refuses to do
|
||||
* silently.
|
||||
*/
|
||||
@Slf4j
|
||||
public final class PdfSensitivityLabels {
|
||||
|
||||
/** Captures the GUID and the attribute name out of {@code MSIP_Label_<guid>_<attr>}. */
|
||||
private static final Pattern LABEL_KEY =
|
||||
Pattern.compile("^MSIP_Label_([0-9a-fA-F-]{36})_(\\w+)$");
|
||||
|
||||
/** Finds the same keys inside a raw XMP packet, whatever schema wraps them. */
|
||||
private static final Pattern XMP_LABEL_ENTRY =
|
||||
Pattern.compile(
|
||||
"<([\\w-]+:)?(MSIP_Label_[0-9a-fA-F-]{36}_\\w+)>([^<]*)</\\1?\\2>",
|
||||
Pattern.CASE_INSENSITIVE);
|
||||
|
||||
/**
|
||||
* Adobe's extension schema for carrying arbitrary Document Info entries in XMP. Using it keeps
|
||||
* the XMP copy standards-shaped instead of inventing a namespace.
|
||||
*/
|
||||
private static final String PDFX_NAMESPACE = "http://ns.adobe.com/pdfx/1.3/";
|
||||
|
||||
private static final int MAX_XMP_BYTES = 8 * 1024 * 1024;
|
||||
|
||||
private PdfSensitivityLabels() {}
|
||||
|
||||
/**
|
||||
* The label on this document, if any.
|
||||
*
|
||||
* <p>A document carries at most one label per organisation, but may carry labels from several.
|
||||
* When more than one is present the first found is returned - callers that care about a
|
||||
* specific tenant should compare {@link SensitivityLabel#siteId()}.
|
||||
*/
|
||||
public static Optional<SensitivityLabel> read(PDDocument document) {
|
||||
List<SensitivityLabel> all = readAll(document);
|
||||
return all.isEmpty() ? Optional.empty() : Optional.of(all.get(0));
|
||||
}
|
||||
|
||||
/** Every label on the document, across both metadata surfaces, de-duplicated by GUID. */
|
||||
public static List<SensitivityLabel> readAll(PDDocument document) {
|
||||
Map<String, Map<String, String>> byLabelId = new LinkedHashMap<>();
|
||||
collect(infoPairs(document), byLabelId);
|
||||
collect(xmpPairs(document), byLabelId);
|
||||
|
||||
List<SensitivityLabel> labels = new ArrayList<>();
|
||||
byLabelId.forEach(
|
||||
(labelId, attributes) -> {
|
||||
SensitivityLabel label = SensitivityLabel.fromAttributes(labelId, attributes);
|
||||
if (label != null) {
|
||||
labels.add(label);
|
||||
}
|
||||
});
|
||||
return labels;
|
||||
}
|
||||
|
||||
/**
|
||||
* Apply a label, replacing any the same tenant already set.
|
||||
*
|
||||
* @throws IllegalArgumentException if the label claims encryption, which this cannot honour
|
||||
*/
|
||||
public static void apply(PDDocument document, SensitivityLabel label) throws IOException {
|
||||
if (label.isProtected()) {
|
||||
// Writing ContentBits=ENCRYPT onto an unencrypted file would tell every downstream
|
||||
// reader the content is protected when it is plaintext. Refuse rather than lie.
|
||||
throw new IllegalArgumentException(
|
||||
"This label requires encryption, which needs the Microsoft Purview client or"
|
||||
+ " MIP SDK; Stirling can apply the label metadata but cannot protect"
|
||||
+ " the content.");
|
||||
}
|
||||
// "An object can only have one label from the same organization." Replace this tenant's
|
||||
// labels on both surfaces, but leave other tenants' labels untouched on both.
|
||||
Set<String> replaced = labelIdsOfTenant(document, label.siteId());
|
||||
replaced.add(label.labelId());
|
||||
Map<String, String> pairs = label.toMetadata();
|
||||
removeInfoLabels(document, replaced::contains);
|
||||
writeInfo(document, pairs);
|
||||
writeXmp(document, pairs, replaced::contains);
|
||||
}
|
||||
|
||||
/** Strip every label, e.g. before re-labelling or when downgrading a document. */
|
||||
public static void clear(PDDocument document) throws IOException {
|
||||
removeInfoLabels(document, labelId -> true);
|
||||
writeXmp(document, Map.of(), labelId -> true);
|
||||
}
|
||||
|
||||
/** The GUIDs of labels this tenant already set, so both surfaces can drop exactly those. */
|
||||
private static Set<String> labelIdsOfTenant(PDDocument document, String siteId) {
|
||||
Set<String> ids = new LinkedHashSet<>();
|
||||
for (SensitivityLabel existing : readAll(document)) {
|
||||
if (siteId.equalsIgnoreCase(existing.siteId())) {
|
||||
ids.add(existing.labelId());
|
||||
}
|
||||
}
|
||||
return ids;
|
||||
}
|
||||
|
||||
/** Drop info-dictionary label entries whose GUID the predicate selects. */
|
||||
private static void removeInfoLabels(PDDocument document, Predicate<String> removeLabelId) {
|
||||
PDDocumentInformation info = document.getDocumentInformation();
|
||||
for (String key : new ArrayList<>(info.getMetadataKeys())) {
|
||||
Matcher matcher = LABEL_KEY.matcher(key);
|
||||
if (matcher.matches() && removeLabelId.test(matcher.group(1))) {
|
||||
info.setCustomMetadataValue(key, null);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private static void writeInfo(PDDocument document, Map<String, String> pairs) {
|
||||
PDDocumentInformation info = document.getDocumentInformation();
|
||||
pairs.forEach(info::setCustomMetadataValue);
|
||||
}
|
||||
|
||||
/**
|
||||
* Rewrite the XMP packet's label properties, leaving the rest of the packet untouched.
|
||||
*
|
||||
* <p>The packet is edited textually rather than re-serialised through xmpbox: a document's XMP
|
||||
* may carry schemas xmpbox does not model, and a round-trip through it would silently drop
|
||||
* them.
|
||||
*/
|
||||
private static void writeXmp(
|
||||
PDDocument document, Map<String, String> pairs, Predicate<String> removeLabelId)
|
||||
throws IOException {
|
||||
PDDocumentCatalog catalog = document.getDocumentCatalog();
|
||||
String existing = readXmpString(catalog);
|
||||
if (existing == null) {
|
||||
if (pairs.isEmpty()) {
|
||||
return;
|
||||
}
|
||||
existing = emptyPacket();
|
||||
}
|
||||
String stripped = stripLabels(existing, removeLabelId);
|
||||
String updated = insertLabelProperties(stripped, pairs);
|
||||
if (updated == null) {
|
||||
log.debug("XMP packet has no rdf:Description to hold the label; info dictionary only");
|
||||
return;
|
||||
}
|
||||
PDMetadata metadata = new PDMetadata(document);
|
||||
metadata.importXMPMetadata(updated.getBytes(StandardCharsets.UTF_8));
|
||||
catalog.setMetadata(metadata);
|
||||
}
|
||||
|
||||
/** Remove only the XMP label entries whose GUID the predicate selects, keeping the rest. */
|
||||
private static String stripLabels(String packet, Predicate<String> removeLabelId) {
|
||||
Matcher matcher = XMP_LABEL_ENTRY.matcher(packet);
|
||||
StringBuilder out = new StringBuilder();
|
||||
while (matcher.find()) {
|
||||
Matcher key = LABEL_KEY.matcher(matcher.group(2));
|
||||
boolean remove = key.matches() && removeLabelId.test(key.group(1));
|
||||
matcher.appendReplacement(out, Matcher.quoteReplacement(remove ? "" : matcher.group()));
|
||||
}
|
||||
matcher.appendTail(out);
|
||||
return out.toString();
|
||||
}
|
||||
|
||||
/** Splice the properties into the first {@code rdf:Description}; null when there is none. */
|
||||
private static String insertLabelProperties(String packet, Map<String, String> pairs) {
|
||||
if (pairs.isEmpty()) {
|
||||
return packet;
|
||||
}
|
||||
Matcher description = Pattern.compile("<rdf:Description\\b[^>]*>").matcher(packet);
|
||||
if (!description.find()) {
|
||||
return null;
|
||||
}
|
||||
StringBuilder properties = new StringBuilder();
|
||||
pairs.forEach(
|
||||
(key, value) ->
|
||||
properties
|
||||
.append("\n <pdfx:")
|
||||
.append(key)
|
||||
.append('>')
|
||||
.append(escapeXml(value))
|
||||
.append("</pdfx:")
|
||||
.append(key)
|
||||
.append('>'));
|
||||
String opening = description.group();
|
||||
String withNamespace =
|
||||
opening.contains("xmlns:pdfx=")
|
||||
? opening
|
||||
: opening.substring(0, opening.length() - 1)
|
||||
+ " xmlns:pdfx=\""
|
||||
+ PDFX_NAMESPACE
|
||||
+ "\">";
|
||||
return packet.substring(0, description.start())
|
||||
+ withNamespace
|
||||
+ properties
|
||||
+ packet.substring(description.end());
|
||||
}
|
||||
|
||||
private static Map<String, String> infoPairs(PDDocument document) {
|
||||
Map<String, String> pairs = new LinkedHashMap<>();
|
||||
PDDocumentInformation info = document.getDocumentInformation();
|
||||
for (String key : info.getMetadataKeys()) {
|
||||
String value = info.getCustomMetadataValue(key);
|
||||
if (value != null) {
|
||||
pairs.put(key, value);
|
||||
}
|
||||
}
|
||||
return pairs;
|
||||
}
|
||||
|
||||
private static Map<String, String> xmpPairs(PDDocument document) {
|
||||
Map<String, String> pairs = new LinkedHashMap<>();
|
||||
String packet;
|
||||
try {
|
||||
packet = readXmpString(document.getDocumentCatalog());
|
||||
} catch (IOException e) {
|
||||
log.debug(
|
||||
"Unreadable XMP packet; falling back to the info dictionary: {}",
|
||||
e.getMessage());
|
||||
return pairs;
|
||||
}
|
||||
if (packet == null) {
|
||||
return pairs;
|
||||
}
|
||||
Matcher matcher = XMP_LABEL_ENTRY.matcher(packet);
|
||||
while (matcher.find()) {
|
||||
pairs.put(matcher.group(2), unescapeXml(matcher.group(3).trim()));
|
||||
}
|
||||
return pairs;
|
||||
}
|
||||
|
||||
/** Group raw pairs by label GUID, keeping the attribute name as the key. */
|
||||
private static void collect(Map<String, String> pairs, Map<String, Map<String, String>> into) {
|
||||
pairs.forEach(
|
||||
(key, value) -> {
|
||||
Matcher matcher = LABEL_KEY.matcher(key);
|
||||
if (!matcher.matches()) {
|
||||
return;
|
||||
}
|
||||
into.computeIfAbsent(matcher.group(1), id -> new LinkedHashMap<>())
|
||||
// Info-dictionary pairs are collected first and win: a stale XMP copy
|
||||
// must not override the value the labelling client wrote.
|
||||
.putIfAbsent(matcher.group(2), value);
|
||||
});
|
||||
}
|
||||
|
||||
private static String readXmpString(PDDocumentCatalog catalog) throws IOException {
|
||||
PDMetadata metadata = catalog.getMetadata();
|
||||
if (metadata == null) {
|
||||
return null;
|
||||
}
|
||||
try (InputStream is = metadata.exportXMPMetadata()) {
|
||||
ByteArrayOutputStream buffer = new ByteArrayOutputStream();
|
||||
byte[] chunk = new byte[8192];
|
||||
int read;
|
||||
int total = 0;
|
||||
while ((read = is.read(chunk)) != -1) {
|
||||
total += read;
|
||||
if (total > MAX_XMP_BYTES) {
|
||||
// A hostile document could otherwise hand us an unbounded packet to hold.
|
||||
throw new IOException("XMP packet exceeds " + MAX_XMP_BYTES + " bytes");
|
||||
}
|
||||
buffer.write(chunk, 0, read);
|
||||
}
|
||||
return buffer.toString(StandardCharsets.UTF_8);
|
||||
}
|
||||
}
|
||||
|
||||
private static String emptyPacket() {
|
||||
return "<?xpacket begin=\"\" id=\"W5M0MpCehiHzreSzNTczkc9d\"?>"
|
||||
+ "<x:xmpmeta xmlns:x=\"adobe:ns:meta/\">"
|
||||
+ "<rdf:RDF xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">"
|
||||
+ "<rdf:Description rdf:about=\"\"></rdf:Description>"
|
||||
+ "</rdf:RDF></x:xmpmeta><?xpacket end=\"w\"?>";
|
||||
}
|
||||
|
||||
private static String escapeXml(String value) {
|
||||
return value.replace("&", "&")
|
||||
.replace("<", "<")
|
||||
.replace(">", ">")
|
||||
.replace("\"", """);
|
||||
}
|
||||
|
||||
private static String unescapeXml(String value) {
|
||||
return value.replace("<", "<")
|
||||
.replace(">", ">")
|
||||
.replace(""", "\"")
|
||||
.replace("&", "&");
|
||||
}
|
||||
}
|
||||
+94
@@ -0,0 +1,94 @@
|
||||
package stirling.software.proprietary.integration.purview;
|
||||
|
||||
import java.util.Locale;
|
||||
import java.util.Map;
|
||||
import java.util.regex.Pattern;
|
||||
|
||||
/**
|
||||
* A Microsoft Purview tenant connection.
|
||||
*
|
||||
* <p>Only {@code tenantId} is required, because labelling a document needs nothing else: a label is
|
||||
* a set of key/value pairs and the tenant id is the {@code SiteId} among them. No call to Microsoft
|
||||
* is involved, so the step works with no network and no app registration.
|
||||
*
|
||||
* <p>The app-registration fields are optional and buy exactly one thing: reading the tenant's label
|
||||
* taxonomy from Graph, so the UI can offer a list of labels instead of asking someone to paste a
|
||||
* GUID. They are not needed to apply or read a label. Graph cannot apply labels for an application
|
||||
* anyway - "application permissions are not supported when updating assignedLabels" - which is why
|
||||
* labelling here goes through the published metadata contract instead.
|
||||
*/
|
||||
public record PurviewConnectionSettings(
|
||||
String tenantId,
|
||||
String clientId,
|
||||
String clientSecret,
|
||||
String graphBaseUrl,
|
||||
String loginBaseUrl) {
|
||||
|
||||
static final String TENANT_ID_OPTION = "tenantId";
|
||||
static final String CLIENT_ID_OPTION = "clientId";
|
||||
// Contains a SecretMasker hint, so it masks on read and merges on update.
|
||||
static final String CLIENT_SECRET_OPTION = "clientSecret";
|
||||
static final String GRAPH_BASE_URL_OPTION = "graphBaseUrl";
|
||||
static final String LOGIN_BASE_URL_OPTION = "loginBaseUrl";
|
||||
|
||||
public static final String DEFAULT_GRAPH_BASE_URL = "https://graph.microsoft.com";
|
||||
public static final String DEFAULT_LOGIN_BASE_URL = "https://login.microsoftonline.com";
|
||||
|
||||
/** Entra tenant ids are GUIDs; the value ends up in document metadata, so it is checked. */
|
||||
private static final Pattern GUID =
|
||||
Pattern.compile("^[0-9a-fA-F]{8}(-[0-9a-fA-F]{4}){3}-[0-9a-fA-F]{12}$");
|
||||
|
||||
public static PurviewConnectionSettings from(Map<String, Object> options) {
|
||||
String tenantId = trimmed(options.get(TENANT_ID_OPTION));
|
||||
if (tenantId == null) {
|
||||
throw new IllegalArgumentException("purview config requires a 'tenantId'");
|
||||
}
|
||||
if (!GUID.matcher(tenantId).matches()) {
|
||||
throw new IllegalArgumentException(
|
||||
"purview config 'tenantId' must be a GUID, e.g."
|
||||
+ " cb46c030-1825-4e81-a295-151c039dbf02");
|
||||
}
|
||||
String clientId = trimmed(options.get(CLIENT_ID_OPTION));
|
||||
String clientSecret = trimmed(options.get(CLIENT_SECRET_OPTION));
|
||||
// Half an app registration would fail only when someone opened the label picker, which is
|
||||
// a confusing place to discover it.
|
||||
if ((clientId == null) != (clientSecret == null)) {
|
||||
throw new IllegalArgumentException(
|
||||
"purview config needs both 'clientId' and 'clientSecret' to read the label"
|
||||
+ " list, or neither");
|
||||
}
|
||||
return new PurviewConnectionSettings(
|
||||
tenantId.toLowerCase(Locale.ROOT),
|
||||
clientId,
|
||||
clientSecret,
|
||||
orDefault(trimmed(options.get(GRAPH_BASE_URL_OPTION)), DEFAULT_GRAPH_BASE_URL),
|
||||
orDefault(trimmed(options.get(LOGIN_BASE_URL_OPTION)), DEFAULT_LOGIN_BASE_URL));
|
||||
}
|
||||
|
||||
/** Whether this connection can read the tenant's label taxonomy from Graph. */
|
||||
public boolean canListLabels() {
|
||||
return clientId != null && clientSecret != null;
|
||||
}
|
||||
|
||||
private static String orDefault(String value, String fallback) {
|
||||
return value == null ? fallback : value;
|
||||
}
|
||||
|
||||
private static String trimmed(Object value) {
|
||||
if (value == null) {
|
||||
return null;
|
||||
}
|
||||
String text = value.toString().trim();
|
||||
return text.isEmpty() ? null : text;
|
||||
}
|
||||
|
||||
/** Never prints the client secret, so an accidental log line cannot leak it. */
|
||||
@Override
|
||||
public String toString() {
|
||||
return "PurviewConnectionSettings[tenantId="
|
||||
+ tenantId
|
||||
+ ", canListLabels="
|
||||
+ canListLabels()
|
||||
+ "]";
|
||||
}
|
||||
}
|
||||
+23
@@ -0,0 +1,23 @@
|
||||
package stirling.software.proprietary.integration.purview;
|
||||
|
||||
import java.util.Map;
|
||||
|
||||
import org.springframework.stereotype.Component;
|
||||
|
||||
import stirling.software.proprietary.integration.model.IntegrationType;
|
||||
import stirling.software.proprietary.integration.service.IntegrationConfigValidator;
|
||||
|
||||
/** The Purview connection schema, enforced when the config is saved. */
|
||||
@Component
|
||||
public class PurviewIntegrationValidator implements IntegrationConfigValidator {
|
||||
|
||||
@Override
|
||||
public IntegrationType type() {
|
||||
return IntegrationType.PURVIEW;
|
||||
}
|
||||
|
||||
@Override
|
||||
public void validate(Map<String, Object> config) {
|
||||
PurviewConnectionSettings.from(config);
|
||||
}
|
||||
}
|
||||
+184
@@ -0,0 +1,184 @@
|
||||
package stirling.software.proprietary.integration.purview;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.time.Instant;
|
||||
import java.util.List;
|
||||
import java.util.Optional;
|
||||
|
||||
import org.apache.pdfbox.pdmodel.PDDocument;
|
||||
import org.springframework.core.io.ByteArrayResource;
|
||||
import org.springframework.core.io.Resource;
|
||||
import org.springframework.http.HttpHeaders;
|
||||
import org.springframework.http.MediaType;
|
||||
import org.springframework.http.ResponseEntity;
|
||||
import org.springframework.web.bind.annotation.PostMapping;
|
||||
import org.springframework.web.bind.annotation.RequestMapping;
|
||||
import org.springframework.web.bind.annotation.RequestParam;
|
||||
import org.springframework.web.bind.annotation.RestController;
|
||||
import org.springframework.web.multipart.MultipartFile;
|
||||
|
||||
import io.github.pixee.security.Filenames;
|
||||
import io.swagger.v3.oas.annotations.Operation;
|
||||
import io.swagger.v3.oas.annotations.tags.Tag;
|
||||
|
||||
import lombok.RequiredArgsConstructor;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
import stirling.software.common.service.CustomPDFDocumentFactory;
|
||||
import stirling.software.common.util.TempFileManager;
|
||||
import stirling.software.common.util.WebResponseUtils;
|
||||
import stirling.software.proprietary.integration.api.ApiConnectionResolver;
|
||||
import stirling.software.proprietary.integration.model.IntegrationType;
|
||||
import stirling.software.proprietary.integration.purview.SensitivityLabel.AssignmentMethod;
|
||||
import stirling.software.proprietary.service.AiToolResponseHeaders;
|
||||
|
||||
import tools.jackson.databind.ObjectMapper;
|
||||
import tools.jackson.databind.node.ArrayNode;
|
||||
import tools.jackson.databind.node.ObjectNode;
|
||||
|
||||
/**
|
||||
* Purview sensitivity labelling as policy steps.
|
||||
*
|
||||
* <p>Both steps are local: a label is metadata, so applying and reading one involves no call to
|
||||
* Microsoft. The connection supplies the tenant id that becomes the label's {@code SiteId}.
|
||||
*
|
||||
* <p>{@code purview-read-label} exists to make labels <em>actionable</em>: it reports what a
|
||||
* document already carries, so a policy can branch on it - the case Purview itself does not cover,
|
||||
* since it labels documents but does not process them.
|
||||
*/
|
||||
@Slf4j
|
||||
@RestController
|
||||
@RequestMapping("/api/v1/integration")
|
||||
@RequiredArgsConstructor
|
||||
@Tag(name = "Integrations", description = "Third-party integration steps.")
|
||||
public class PurviewLabelController {
|
||||
|
||||
private final ApiConnectionResolver connectionResolver;
|
||||
private final CustomPDFDocumentFactory pdfDocumentFactory;
|
||||
private final TempFileManager tempFileManager;
|
||||
private final ObjectMapper objectMapper;
|
||||
|
||||
@PostMapping(value = "/purview-apply-label", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
|
||||
@Operation(
|
||||
summary = "Apply a Microsoft Purview sensitivity label",
|
||||
description =
|
||||
"Writes the Purview label metadata (MSIP_Label_<GUID>_*) onto the PDF, so"
|
||||
+ " Purview-aware tools recognise the label. Applies the label only;"
|
||||
+ " it cannot encrypt, which requires the Microsoft client."
|
||||
+ " Input:PDF Output:PDF Type:SISO")
|
||||
public ResponseEntity<Resource> applyLabel(
|
||||
@RequestParam("fileInput") MultipartFile fileInput,
|
||||
@RequestParam("connectionId") String connectionId,
|
||||
@RequestParam("labelId") String labelId,
|
||||
@RequestParam(value = "labelName", required = false) String labelName,
|
||||
@RequestParam(value = "method", defaultValue = "STANDARD") String method,
|
||||
@RequestParam(value = "contentBits", required = false) Integer contentBits)
|
||||
throws IOException {
|
||||
|
||||
PurviewConnectionSettings settings = settings(connectionId);
|
||||
AssignmentMethod assignment = parseMethod(method);
|
||||
|
||||
try (PDDocument document = pdfDocumentFactory.load(fileInput, true)) {
|
||||
String fileName = safeFileName(fileInput.getOriginalFilename());
|
||||
SensitivityLabel label =
|
||||
new SensitivityLabel(
|
||||
labelId.trim(),
|
||||
labelName,
|
||||
settings.tenantId(),
|
||||
assignment,
|
||||
Instant.now(),
|
||||
contentBits);
|
||||
PdfSensitivityLabels.apply(document, label);
|
||||
log.debug("[purview-apply-label] labelled {} as {}", fileName, labelId);
|
||||
return WebResponseUtils.pdfDocToWebResponse(document, fileName, tempFileManager);
|
||||
}
|
||||
}
|
||||
|
||||
@PostMapping(value = "/purview-read-label", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
|
||||
@Operation(
|
||||
summary = "Read the Microsoft Purview sensitivity label on a PDF",
|
||||
description =
|
||||
"Reports the Purview labels a PDF already carries so a policy can act on"
|
||||
+ " them. The document passes through unchanged."
|
||||
+ " Input:PDF Output:PDF Type:SISO")
|
||||
public ResponseEntity<Resource> readLabel(
|
||||
@RequestParam("fileInput") MultipartFile fileInput,
|
||||
@RequestParam("connectionId") String connectionId)
|
||||
throws IOException {
|
||||
|
||||
PurviewConnectionSettings settings = settings(connectionId);
|
||||
|
||||
List<SensitivityLabel> labels;
|
||||
try (PDDocument document = pdfDocumentFactory.load(fileInput, true)) {
|
||||
labels = PdfSensitivityLabels.readAll(document);
|
||||
}
|
||||
// The document is returned byte-for-byte rather than re-saved: a read must not perturb the
|
||||
// file it inspected, and a PDFBox round-trip would rewrite its structure.
|
||||
byte[] bytes = fileInput.getBytes();
|
||||
HttpHeaders headers = new HttpHeaders();
|
||||
headers.setContentType(MediaType.APPLICATION_PDF);
|
||||
headers.setContentDispositionFormData(
|
||||
"attachment", safeFileName(fileInput.getOriginalFilename()));
|
||||
headers.setContentLength(bytes.length);
|
||||
headers.set(AiToolResponseHeaders.TOOL_REPORT, buildReport(labels, settings));
|
||||
return ResponseEntity.ok().headers(headers).body(new ByteArrayResource(bytes));
|
||||
}
|
||||
|
||||
/**
|
||||
* The labels found, and which of them is this tenant's - a document can carry labels from
|
||||
* several organisations, and only the matching one reflects this tenant's policy.
|
||||
*/
|
||||
private String buildReport(List<SensitivityLabel> labels, PurviewConnectionSettings settings) {
|
||||
Optional<SensitivityLabel> own =
|
||||
labels.stream()
|
||||
.filter(label -> settings.tenantId().equalsIgnoreCase(label.siteId()))
|
||||
.findFirst();
|
||||
ObjectNode report = objectMapper.createObjectNode();
|
||||
report.put("labelled", own.isPresent());
|
||||
own.ifPresent(
|
||||
label -> {
|
||||
report.put("labelId", label.labelId());
|
||||
report.put("labelName", label.name());
|
||||
report.put("method", label.method() == null ? null : label.method().name());
|
||||
report.put(
|
||||
"setDate", label.setDate() == null ? null : label.setDate().toString());
|
||||
report.put("contentBits", label.contentBits());
|
||||
report.put("protected", label.isProtected());
|
||||
});
|
||||
ArrayNode others = report.putArray("otherTenantLabels");
|
||||
labels.stream()
|
||||
.filter(label -> !settings.tenantId().equalsIgnoreCase(label.siteId()))
|
||||
.forEach(
|
||||
label -> {
|
||||
ObjectNode node = others.addObject();
|
||||
node.put("labelId", label.labelId());
|
||||
node.put("siteId", label.siteId());
|
||||
});
|
||||
return objectMapper.writeValueAsString(report);
|
||||
}
|
||||
|
||||
private PurviewConnectionSettings settings(String connectionId) {
|
||||
Long id = ApiConnectionResolver.connectionId(connectionId);
|
||||
if (id == null) {
|
||||
throw new IllegalArgumentException("'connectionId' is required");
|
||||
}
|
||||
return PurviewConnectionSettings.from(
|
||||
connectionResolver.resolveConfig(id, IntegrationType.PURVIEW));
|
||||
}
|
||||
|
||||
private static AssignmentMethod parseMethod(String method) {
|
||||
AssignmentMethod parsed = AssignmentMethod.parse(method);
|
||||
if (parsed == null) {
|
||||
throw new IllegalArgumentException(
|
||||
"'method' must be STANDARD (applied automatically) or PRIVILEGED (chosen by a"
|
||||
+ " person); got "
|
||||
+ method);
|
||||
}
|
||||
return parsed;
|
||||
}
|
||||
|
||||
private static String safeFileName(String originalFilename) {
|
||||
String name = Filenames.toSimpleFileName(originalFilename);
|
||||
return (name == null || name.isBlank()) ? "labelled.pdf" : name;
|
||||
}
|
||||
}
|
||||
+186
@@ -0,0 +1,186 @@
|
||||
package stirling.software.proprietary.integration.purview;
|
||||
|
||||
import java.time.Instant;
|
||||
import java.time.ZoneOffset;
|
||||
import java.time.format.DateTimeFormatter;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.Locale;
|
||||
import java.util.Map;
|
||||
import java.util.regex.Pattern;
|
||||
|
||||
/**
|
||||
* One Microsoft Purview Information Protection label as it is written to a document.
|
||||
*
|
||||
* <p>Microsoft persists a label as a flat set of key/value pairs named {@code
|
||||
* MSIP_Label_<GUID>_<Attribute>}, and documents that contract publicly so third-party software can
|
||||
* read a label and act on it. That published contract - not the MIP SDK, which has no Java binding
|
||||
* - is what this type implements. See <a
|
||||
* href="https://learn.microsoft.com/en-us/information-protection/develop/concept-mip-metadata">Label
|
||||
* metadata in the MIP SDK</a>.
|
||||
*
|
||||
* <p>Only {@code Enabled} and {@code SiteId} are mandatory in that contract; the rest are optional
|
||||
* and may be absent on a label written by an older client, so readers here tolerate their absence.
|
||||
*/
|
||||
public record SensitivityLabel(
|
||||
String labelId,
|
||||
String name,
|
||||
String siteId,
|
||||
AssignmentMethod method,
|
||||
Instant setDate,
|
||||
Integer contentBits) {
|
||||
|
||||
/** How the label came to be applied. */
|
||||
public enum AssignmentMethod {
|
||||
/** Applied by default or automatically - e.g. by a policy like this one. */
|
||||
STANDARD,
|
||||
/** Chosen deliberately by a person. */
|
||||
PRIVILEGED;
|
||||
|
||||
String wireValue() {
|
||||
return name().charAt(0) + name().substring(1).toLowerCase(Locale.ROOT);
|
||||
}
|
||||
|
||||
static AssignmentMethod parse(String value) {
|
||||
if (value == null) {
|
||||
return null;
|
||||
}
|
||||
try {
|
||||
return valueOf(value.trim().toUpperCase(Locale.ROOT));
|
||||
} catch (IllegalArgumentException e) {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public static final String KEY_PREFIX = "MSIP_Label_";
|
||||
|
||||
/** Content marks the labelling application applied; a bitmask, per the MIP contract. */
|
||||
public static final int CONTENT_BITS_HEADER = 0x1;
|
||||
|
||||
public static final int CONTENT_BITS_FOOTER = 0x2;
|
||||
public static final int CONTENT_BITS_WATERMARK = 0x4;
|
||||
public static final int CONTENT_BITS_ENCRYPT = 0x8;
|
||||
|
||||
/**
|
||||
* Extended ISO 8601, matching the {@code 2018-11-08T21:13:16-0800} form Microsoft documents.
|
||||
*/
|
||||
private static final DateTimeFormatter SET_DATE =
|
||||
DateTimeFormatter.ofPattern("yyyy-MM-dd'T'HH:mm:ssZ", Locale.ROOT)
|
||||
.withZone(ZoneOffset.UTC);
|
||||
|
||||
/**
|
||||
* Microsoft caps each key and value at 255 characters "to maintain compatibility across common
|
||||
* applications".
|
||||
*/
|
||||
static final int MAX_VALUE_LENGTH = 255;
|
||||
|
||||
/** The GUID shape a labelId must take, matching what the read path accepts from a document. */
|
||||
private static final Pattern LABEL_ID = Pattern.compile("^[0-9a-fA-F-]{36}$");
|
||||
|
||||
public SensitivityLabel {
|
||||
if (labelId == null || labelId.isBlank()) {
|
||||
throw new IllegalArgumentException("a sensitivity label needs a labelId");
|
||||
}
|
||||
if (!LABEL_ID.matcher(labelId).matches()) {
|
||||
// labelId is spliced verbatim into XMP/info key names; a non-GUID would let a stray
|
||||
// character (a space, or <, >, &) corrupt or inject the metadata it is written into.
|
||||
throw new IllegalArgumentException("a sensitivity label needs a GUID labelId");
|
||||
}
|
||||
if (siteId == null || siteId.isBlank()) {
|
||||
throw new IllegalArgumentException("a sensitivity label needs a siteId (tenant id)");
|
||||
}
|
||||
}
|
||||
|
||||
/** The {@code MSIP_Label_<GUID>_} prefix this label's keys share. */
|
||||
public String keyPrefix() {
|
||||
return KEY_PREFIX + labelId + "_";
|
||||
}
|
||||
|
||||
/**
|
||||
* This label as the key/value pairs to persist. Optional attributes are omitted when unset
|
||||
* rather than written empty, so a reader cannot mistake "not recorded" for "recorded as blank".
|
||||
*/
|
||||
public Map<String, String> toMetadata() {
|
||||
Map<String, String> out = new LinkedHashMap<>();
|
||||
String prefix = keyPrefix();
|
||||
out.put(prefix + "Enabled", "true");
|
||||
out.put(prefix + "SiteId", siteId);
|
||||
if (method != null) {
|
||||
out.put(prefix + "Method", method.wireValue());
|
||||
}
|
||||
if (setDate != null) {
|
||||
out.put(prefix + "SetDate", SET_DATE.format(setDate));
|
||||
}
|
||||
if (name != null && !name.isBlank()) {
|
||||
out.put(prefix + "Name", truncate(name));
|
||||
}
|
||||
if (contentBits != null) {
|
||||
out.put(prefix + "ContentBits", String.valueOf(contentBits));
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* Rebuild a label from the pairs found on a document.
|
||||
*
|
||||
* @param labelId the GUID between the prefix and the attribute name
|
||||
* @param attributes attribute name (e.g. {@code Name}) to value, for that GUID only
|
||||
* @return null when the pairs do not describe an enabled label
|
||||
*/
|
||||
static SensitivityLabel fromAttributes(String labelId, Map<String, String> attributes) {
|
||||
// "DLP products typically validate the existence of this key to identify the
|
||||
// classification label" - an absent or false Enabled means there is no label here.
|
||||
if (!"true".equalsIgnoreCase(attributes.get("Enabled"))) {
|
||||
return null;
|
||||
}
|
||||
String siteId = attributes.get("SiteId");
|
||||
if (siteId == null || siteId.isBlank()) {
|
||||
// SiteId is mandatory in the contract, but a label written by something non-compliant
|
||||
// is still a label; keep it readable rather than throwing on someone else's file.
|
||||
siteId = "unknown";
|
||||
}
|
||||
return new SensitivityLabel(
|
||||
labelId,
|
||||
attributes.get("Name"),
|
||||
siteId,
|
||||
AssignmentMethod.parse(attributes.get("Method")),
|
||||
parseDate(attributes.get("SetDate")),
|
||||
parseInt(attributes.get("ContentBits")));
|
||||
}
|
||||
|
||||
private static Instant parseDate(String value) {
|
||||
if (value == null || value.isBlank()) {
|
||||
return null;
|
||||
}
|
||||
try {
|
||||
return SET_DATE.parse(value.trim(), Instant::from);
|
||||
} catch (RuntimeException e) {
|
||||
try {
|
||||
// Tolerate the plain ISO form some writers use instead.
|
||||
return Instant.parse(value.trim());
|
||||
} catch (RuntimeException ignored) {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private static Integer parseInt(String value) {
|
||||
if (value == null || value.isBlank()) {
|
||||
return null;
|
||||
}
|
||||
try {
|
||||
return Integer.valueOf(value.trim());
|
||||
} catch (NumberFormatException e) {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
private static String truncate(String value) {
|
||||
return value.length() <= MAX_VALUE_LENGTH ? value : value.substring(0, MAX_VALUE_LENGTH);
|
||||
}
|
||||
|
||||
/** Whether the labelling application encrypted the content. */
|
||||
public boolean isProtected() {
|
||||
return contentBits != null && (contentBits & CONTENT_BITS_ENCRYPT) != 0;
|
||||
}
|
||||
}
|
||||
+32
@@ -13,6 +13,7 @@ import org.springframework.web.server.ResponseStatusException;
|
||||
import lombok.RequiredArgsConstructor;
|
||||
import lombok.extern.slf4j.Slf4j;
|
||||
|
||||
import stirling.software.common.model.ApplicationProperties;
|
||||
import stirling.software.proprietary.access.model.DefaultAccessPolicy;
|
||||
import stirling.software.proprietary.access.model.OwnerScope;
|
||||
import stirling.software.proprietary.access.model.ResourceType;
|
||||
@@ -43,6 +44,7 @@ public class IntegrationConfigService {
|
||||
private final OwnershipService ownership;
|
||||
private final SecretMasker secretMasker;
|
||||
private final ResourceGrantRepository grantRepository;
|
||||
private final ApplicationProperties applicationProperties;
|
||||
// Bean-discovered extension points: features that understand a type contribute its config
|
||||
// schema and report what still references a config, without this module depending on them.
|
||||
private final List<IntegrationConfigValidator> validators;
|
||||
@@ -62,6 +64,7 @@ public class IntegrationConfigService {
|
||||
&& !ownership.isAdmin(currentUser)) {
|
||||
throw forbidden("S3 connections can only be created by administrators or team owners");
|
||||
}
|
||||
requireCustomApiAllowed(cfg.getIntegrationType(), currentUser);
|
||||
cfg.setName(require(request.name(), "name"));
|
||||
cfg.setEnabled(request.enabled() == null || request.enabled());
|
||||
cfg.setLocked(request.locked() != null && request.locked());
|
||||
@@ -113,6 +116,9 @@ public class IntegrationConfigService {
|
||||
cfg.setDefaultAccess(request.defaultAccess());
|
||||
}
|
||||
if (request.config() != null) {
|
||||
// Editing the config of a custom integration is the same authoring power as creating
|
||||
// one - it is where the base URL and body live - so it is gated identically.
|
||||
requireCustomApiAllowed(cfg.getIntegrationType(), currentUser);
|
||||
Map<String, Object> merged =
|
||||
secretMasker.merge(readJson(cfg.getConfig()), request.config());
|
||||
validateConfig(cfg.getIntegrationType(), merged);
|
||||
@@ -121,6 +127,32 @@ public class IntegrationConfigService {
|
||||
return repository.save(cfg);
|
||||
}
|
||||
|
||||
/**
|
||||
* A custom API integration names its own host, path and body, so it can point the server
|
||||
* anywhere. That is authoring power rather than self-serve configuration: admins only, and the
|
||||
* operator can withdraw it entirely. The vendor presets are not gated here - they carry a fixed
|
||||
* shape, so the worst a user can do is misconfigure their own connection.
|
||||
*/
|
||||
private void requireCustomApiAllowed(IntegrationType type, User currentUser) {
|
||||
if (type != IntegrationType.API) {
|
||||
return;
|
||||
}
|
||||
if (!applicationProperties.getPolicies().isAllowCustomApiIntegrations()) {
|
||||
throw forbidden(
|
||||
"Custom API integrations are disabled on this server"
|
||||
+ " (policies.allowCustomApiIntegrations)");
|
||||
}
|
||||
if (!ownership.isAdmin(currentUser)) {
|
||||
throw forbidden("Custom API integrations can only be created by administrators");
|
||||
}
|
||||
}
|
||||
|
||||
/** Whether this caller may author custom API integrations, for the UI to offer or hide it. */
|
||||
public boolean canAuthorCustomApi(User currentUser) {
|
||||
return applicationProperties.getPolicies().isAllowCustomApiIntegrations()
|
||||
&& ownership.isAdmin(currentUser);
|
||||
}
|
||||
|
||||
@Transactional
|
||||
public void delete(Long id, User currentUser) {
|
||||
IntegrationConfig cfg = load(id);
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user