Compare commits

...
15 Commits
Author SHA1 Message Date
Anthony Stirling 84ac22a7de Generate the OpenAPI spec on a free port (#7777)
`generateOpenApiDocs` forks a Spring Boot instance to harvest the spec,
but the scrape URL was hardcoded to `http://localhost:8080` - the app's
own port. If any Stirling instance is already listening there (a dev
server, a container, a WSL relay) the plugin reads **that** build
instead of the fork, and `task tool-models` then generates the Python
and TypeScript models from whatever happens to be running.

It fails silently and looks like success. I hit it while regenerating
models after editing Java annotations: the build passed, the compiled
class contained the new text, and the spec contained none of it.
Fetching the URL by hand settled it:

```
paths live8080: 276   ours: 276
identical parsed content: True
```

The generated `SwaggerDoc.json` was the running instance's document, not
the working tree's.

Now on a random free port, passed to both the fork and the scrape URL so
they cannot disagree. Override with `-PopenApiPort=NNNN` to use a
specific port.
2026-09-03 09:41:55 +00:00
Anthony Stirling 70cdf2429f Put the user request last in the planner prompt (#7773)
Two changes to the `pdf_edit` selection prompt.

**The user's request moves to the end.** When a prompt exceeds the
model's context, Ollama keeps the tail and discards the head. The
request was the second line, so it was the first thing thrown away.
Ending with it makes the most important line the most likely to survive.

Reproduced on a local qwen3:8b before the change: five completely
different requests - convert to Word, merge two files, run OCR, add a
password, add a watermark - run sequentially at temperature 0 through
the production prompt shape all returned `COMPRESS_PDF`, and every one
reported `prompt_tokens: 2050` against a much larger prompt. The model
was not choosing badly; it was answering a question it had never been
shown.

This is defensive rather than a fix on its own - the real repair is
giving the model a context window large enough, and a prompt small
enough, to avoid truncation entirely. But the ordering is free and
correct regardless of context size.

**The duplicated unavailable-operations list is removed from the user
prompt.** It was rendered twice, once in the system prompt and again in
the user prompt. It is also counterproductive: it spends tokens teaching
the model names it must not use, and `handle()` already rejects a plan
referencing an unavailable operation in Python afterwards. The
system-prompt copy stays, so the model can still explain that an
operation exists but is not available here.

The existing test that asserted the list appears in the user prompt now
asserts it appears in the system prompt and not the user prompt.
2026-09-03 09:41:45 +00:00
ConnorYoh cbc8af1951 feat(desktop): custom in-app window title bar on Windows (#7781)
<img width="1750" height="1223" alt="image"
src="https://github.com/user-attachments/assets/c115a30c-be62-4eaa-8de3-26a93a605919"
/>

## Custom windows top bar
* Doesn't work on mac
* doesn't effect web 
* little effort been put into mobile view
2026-09-02 21:31:45 +00:00
James BruntonandEthanHealy01 aca0e40c37 Combine Policies and Pipelines pages (#7681)
# Description of Changes
Combine the Policies and Pipelines pages into one, so we have the new
concept of Policies as Pipelines that always run which the user cannot
disable. What used to be Policies are now referred to as Templates, and
they allow you to create a new Pipeline more easily with the simple UI.

There's followup work to be done here to improve the template UIs
because they've not been touched in a long time, but I've considered
that beyond the scope of this merge. The only real changes I've made to
them in this PR is that they have a toggle for whether they're policies,
they now have a "Customise" button to kick you into the full Pipeline
editor, and I've removed the source selection. Previously, they
supported selecting as many sources as you liked, but that feature never
worked and is incompatible with the backend as it stands now, which only
allows for one source. Because of that, I've made it so that they can
only run in editor unless you open them in the custom pipeline editor,
where you can switch out which source it will use.

There's also another bit of followup to rename and remove all the
previous Policies code. Now that they've been combined into one, we
don't need a lot of the Policies code anymore, but also there's about
300 files in the frontend referencing policies in text/comments which
need to be updated to say pipelines. This is way more work than is
reasonable to do in this PR so I'll just do it in a new PR.

## Limitations
This PR is about the merging of the old Policies and Pipelines and I'm
considering enforcing the new definition of a Policy where it's only
modifiable by admins beyond the scope of this PR.

<img width="756" height="395" alt="image"
src="https://github.com/user-attachments/assets/d31be5ce-f1c9-46b3-8e8d-866e63f89a81"
/>

<img width="1507" height="793" alt="image"
src="https://github.com/user-attachments/assets/9ba8875f-8be5-4881-91cf-40e0bc1076dc"
/>

<img width="1508" height="787" alt="image"
src="https://github.com/user-attachments/assets/3e1da77b-a0c0-4262-aad3-16650098db81"
/>

---------

Co-authored-by: EthanHealy01 <80844253+EthanHealy01@users.noreply.github.com>
2026-09-02 16:08:15 +00:00
Reece Browne 1b2a3118a6 Disk-mounted folders on desktop and improved folder management (#7502)
Description of Changes

Adds folder kinds so the file manager can work with real directories on
disk.

Desktop
- New folder is now a menu with two options: "Add local folder" and "New
folder on the server".
- Add local folder opens the native picker and mounts a directory. Files
are listed straight from disk, nothing is copied in.
- Subfolders show inside a mount and open like any folder. New folder
inside a mount creates a real directory on disk.
- Moving, dropping or uploading files into a mount writes them to the
directory. The app copy is only removed after the write succeeds. Name
clashes get a " (2)" suffix.
- Mounted files get thumbnails.
- Adding the same directory twice just returns the existing mount.
- Removing a mount never touches the disk.
- The server option is disabled in local mode with a sign in message.

Web + desktop
- Uploading or dropping files while inside a folder puts them in that
folder instead of Local.
- Files can be dragged onto folders in the grid and the tree to move
them.
- Folders show an origin badge (cloud or local).
- The Local view now means files that are not in any folder.

Follow ups for a future pr
- Mount listing cap: large directories currently show the 500 most
recent files with no notice. Will be removed as part of the
virtualisation/performance PR.
- Folders within folders need to be supported
- Symlinks in mounts: currently not listed. Behaviour to be decided
alongside the wider folder work.
2026-09-02 14:18:57 +00:00
ConnorYoh 3056e5ff44 Reset the PAYG free grant each billing period (#7709)
Needs the schema half: Stirling-Tools/Stirling-PDF-SaaS#327

## Current state

The PAYG free allowance is a one-time lifetime pool.
`pricing_policy.free_tier_units` is copied into
`payg_team_extensions.free_units_remaining` once, at team creation (V14
trigger, updated in V19), and the charge pipeline decrements it until it
reaches zero. Nothing ever puts it back.

## Problem

The product promises a monthly allowance the billing model does not
grant.

- The account-link connect dialog advertises "500 free per month". That
has **merged to main** (#7415), so the claim is live and unhonoured
until this lands.
- The wallet meter already read "Process 500 PDFs free, then $X/PDF",
which reads as an allowance-then-meter model.
- `SignupRequiredBootstrap`'s own doc comment described a "free
500-op/month allowance" while its copy said only "500 free operations".

Three separate comments asserted the opposite in code
(`billing/types.ts`, `WalletSnapshotResponse`, `TeamBillingContext`), so
the two halves of the repo disagreed about what a customer is owed.

## Solution

The grant now recurs each billing period, **for every team**. Paying
does not cost you the allowance: a subscribed team draws its grant first
each period and meters only the excess, which is what the meter's copy
always described. That also matches how the grant already worked at
charge time, where it reduced metered units regardless of subscription.

### The reset is lazy, with no scheduler

`payg_team_extensions` gains `free_units_period_start`: the period
`free_units_remaining` was last written for.

- A stamp older than the current period start, or absent as on every
existing row, means the reset is owed.
`TeamBillingService.remainingForPeriod` projects it to a full grant, so
the entitlement gate and the wallet both show it the instant the period
turns.
- `JobChargeService.consumeFreeGrant` persists it on the next charge,
under the pessimistic row lock that already makes the per-job free/paid
split exact.

One rule, both callers, so display and enforcement cannot drift onto
separate schedules. A team that runs nothing for a month has nothing to
write, and no job is needed to hand out the grant.

### One period definition

"Per period" is `TeamBillingContext.periodStart`: the Stripe
subscription's current period when subscribed, the calendar month
otherwise. It was already the only period notion in the system, so the
grant joined it rather than inventing its own:

- `InstanceEntitlement.periodCapUnits` is enforced over the same window.
- `localUsageService.currentPeriodUnsynced` already buckets a linked
instance's local usage by the `periodStart` it reads from the same
snapshot, and resets its counters on that boundary.

For an un-subscribed team, the only kind the grant gates, that window is
the calendar month, which is what the copy promises.

The period rule stays in Java by choice, not necessity: SQL could reach
the Stripe period through the sync engine, but restating the rule there
would give it a second home to drift from. Hence a nullable column and
no backfill in the migration — NULL already means "stale", so every
existing team reads as owed the current period's grant.

### Refunds

A refund landing after the period turned would have stacked last
period's units on top of the fresh grant.
`JobChargeService.restoreFreeGrant` now clamps the restore to one
period's grant, taking the same row lock, and the bulk-increment
`restoreFreeUnits` query is gone. Removing it also removed a `@Query`
string that no test would have parsed before application startup.

### Copy and comments

Every comment and user-facing string that asserted the lifetime model is
corrected. The strings that changed (code defaults and `en-US` TOML
updated together):

| Key | Now reads |
| --- | --- |
| `portal.billing.walletMeter.title` / `titleWithRate` | "500 free
credits every month, then $X per PDF" |
| `portal.billing.walletMeter.capSuffix` / `barAria` | "of 500 free
credits left this month" / "Free credits remaining" |
| `payg.free.hero.capSuffix` | "of 500 free PDFs left this month" |
| `plan.freeLimit.message` | "...this month. ... It resets next month,
or keep the momentum going now..." |
| `payg.signupRequired.body` | "500 free operations a month" |

Main rewrote these keys to "500 free credits to start" while this branch
was open. The merge keeps main's credits vocabulary and drops "to
start", which asserts the one-time grant this branch removes and which
main's own connect dialog already contradicts.

Also fixed in passing: `testing/compose/payg/saas-seed.sql` still
inserted `free_tier_units_per_cycle`, the pre-V19 column name, so that
INSERT had been failing since the rename.

## How to test

Backend:

```bash
STIRLING_FLAVOR=saas ./gradlew :saas:test spotlessCheck
```

Frontend:

```bash
task frontend:typecheck && task frontend:lint && task frontend:format:check
```

New coverage, 10 tests:

- `TeamBillingServiceMoreTest` — a past-period stamp reads as a fresh
grant, a current stamp reads the stored balance, an unstamped row reads
as a fresh grant, the grant follows the Stripe window rather than the
calendar month, plus the `remainingForPeriod` rule itself including a
future stamp and null/negative balances.
- `JobChargeServiceTest` — the first charge of a new period resets and
re-stamps, an unstamped row resets, a zero-grant policy still advances
the stamp, and a refund crossing a period boundary does not exceed the
grant.

Manually, against a team whose grant is spent: set
`free_units_period_start` back a month (or leave it NULL) and the
wallet, the sidebar meter and the entitlement gate should all show a
full grant before any job runs. The first billable job should then draw
from it and write the reset.

Three tests fail on a local Windows run and pass in CI, on files this
branch does not touch: `workbenchSession.test.ts`,
`notificationActions.test.tsx`, and `:proprietary`
`FolderIdentitiesTest.identityAgreesAcrossASymlinkedAliasOfTheDirectory`.
Nothing to do here — noted so a local run does not look like a
regression.

## Merge order

The migration is additive, and Hibernate `ddl-auto=update` will add the
column in a dev environment, so either order works locally. Beyond that
the schema goes first: Stirling-Tools/Stirling-PDF-SaaS#327 targets `v3`
(staging), so it needs to reach an environment before this lands there.
2026-09-02 13:57:56 +00:00
Anthony Stirling 798ba57f0b Reply to chat in the user's UI language (#7766)
# Description of Changes

Pass a user browser lang ID to engine

<img width="1400" height="900" alt="image"
src="https://github.com/user-attachments/assets/7e8fc5c2-8881-4a74-b718-7f5cd350d457"
/>



---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] Every comment I added says something the code does not
([guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/CODE_COMMENTS.md))
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-09-02 12:35:22 +00:00
Anthony Stirling 42bdce155c Fix mobile scanner upload flow and fit it to one screen (#7684)
file mobile phone scanner UI issues when on http and scaling UI issues
Ensuring that smaller screens dont cut off UI elements 
better handling of batch photos

<img width="2104" height="8800" alt="montage_mobile-scanner"
src="https://github.com/user-attachments/assets/b4dd114b-c54d-4101-8700-7307dbb0eee9"
/>


---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-09-02 09:13:31 +00:00
ConnorYoh 2cf355c5cd feat(editor): move admin settings onto TanStack Query (#7437)
# Description of Changes

## The problem

`useAdminSettings` backs all 18 admin config sections. Each section
fetched its own copy of its settings block, held it in hand-rolled
loading/saving state, and refetched manually after every save.

Three consequences:

- **Duplicate fetching.** Four AI tabs all read the `aiEngine` block.
Nothing was shared, so each open refetched it.
- **Duplicated wiring.** All 18 sections carried the same effect to
trigger the fetch, each one depending on a `fetchSettings` callback that
would have refetched on every render had it ever become unstable.
- **Console noise.** The hook made 11 `console.*` calls, four of them
`JSON.stringify(settings, null, 2)` on **every fetch and every save** —
admin configuration serialised into the console of every admin session.

Every save also ended with a hand-written `await fetchSettings()`.
Forget it in a new section and its pending badges silently go stale.

## The fix

The hook uses TanStack Query, keyed on `sectionName`, so sections
reading the same block share one fetch and one cache entry.

The fetch gate moved into the hook. Sections used to write:

```ts
const { settings, fetchSettings } = useAdminSettings({ sectionName: "legal" });

useEffect(() => {
  if (loginEnabled) fetchSettings();
}, [loginEnabled, fetchSettings]);
```

and now write:

```ts
const { settings } = useAdminSettings({
  sectionName: "legal",
  enabled: loginEnabled,
});
```

Saving is a mutation that invalidates the section on success, so the
refetch is structural rather than something each section remembers.

The delta computation and the save transformer are unchanged — that is
domain logic, not fetching. `settings` is still an editable draft seeded
from the server response, so forms behave exactly as before.

## Why it is better

Measured against the previous implementation across identical scenarios.
`commits` counts committed renders.

| Scenario | Before | After |
|---|---|---|
| Open one section | 2 commits, 1 request | 2 commits, 1 request |
| Browse the four AI tabs | 8 commits, 4 requests | **5 commits, 1
request** |
| Edit and save | 4 commits, 2 requests | 4 commits, 2 requests |

Committed renders are equal or better everywhere; browsing the AI tabs
costs a quarter of the requests.

The diff reads +449 / −303, but that includes a test file for a hook
that had no tests:

| | Added | Removed | Net |
|---|---|---|---|
| Production code (21 files) | 154 | 303 | **−149** |
| Tests (1 file) | 295 | 0 | +295 |

The 18 section files account for −133 of that: each drops an effect, a
destructure and usually an import, and gains one `enabled:` line. The
hook itself goes from 234 to 180 lines. `console.*` calls go from 11 to
0.

## Caching

Settings inherit the client's 30s stale window rather than refetching on
every mount, which is where the request saving comes from.

Nothing inside a cached block is server-observed — the only live reads
in these sections, `/api/v1/ai/health` and the tessdata language list,
are separate calls outside this query. A block therefore only changes
when another admin writes it.

Two things bound the staleness:

- Sections already held a single snapshot for as long as the modal
stayed open, with no refetch on focus. 30s is shorter than that window,
not longer.
- `computeDelta` only emits fields whose draft differs from the baseline
it was seeded from, so a stale baseline cannot produce a collateral
write. The only race is two admins editing the same field, which is
unchanged. Saving invalidates, so acting refreshes to current values.

The blocks where a stale read would matter most — `security`, `premium`,
`database` — are set once at deployment and effectively never edited
concurrently. The block with the most cache reuse, `aiEngine`, is the
least consequential.

**Convention:** config blocks cache; observed state does not. A section
that displays live server state inside its settings block should
override `staleTime` locally.

## Testing

14 tests, covering the shared fetch, cache reuse across tab reopens, key
separation between blocks, the `enabled` gate, delta-only saves, the
empty-delta short circuit, post-save invalidation, pending-value
display, and draft reseeding.

Each was checked by breaking the implementation and confirming the suite
fails: per-consumer query keys, sending the whole draft instead of the
delta, dropping the post-save invalidate, reporting loaded while
disabled, skipping the empty-delta short circuit, and reverting the
stale window to zero.

`task frontend:check` green. Two unrelated tests fail on this branch —
`workbenchSession.test.ts` and `notificationActions.test.tsx` — and fail
identically on `main`.

## Follow-ups

The sections that fetch through services rather than this hook — Teams,
TeamDetails, People, roughly 2,600 lines — are unchanged. Between them
they share two reads (`getTeams` and `getUsers`, both used by all three)
and carry ten distinct write operations, with no test coverage today.

---

## Primer: mutations

`useQuery` is for reads. It caches, dedupes, and re-renders when data
arrives. `useMutation` is for writes, where none of that applies — a
write happens once, when the user asks.

```ts
const save = useMutation({
  mutationFn: (body) => putAdminSection("legal", body),
  onSuccess: () => queryClient.invalidateQueries({ queryKey }),
});

save.mutate(body);            // fire and forget
await save.mutateAsync(body); // or await it
save.isPending;               // disable the button
save.error;                   // show the failure
```

`isPending` and `error` replace the `useState` flag and
`try/catch/finally` you would otherwise write around every save.

After a write the cache holds stale data. Two ways to fix it:

| | What it does | Use when |
|---|---|---|
| `invalidateQueries` | Marks the data stale so it refetches | The
server may transform, queue or reject part of what you sent |
| `setQueryData` | Writes your value into the cache, no request | The
response tells you exactly what the server now holds |

**Invalidate by default. Use `setQueryData` only when the response is
authoritative.**

This hook has to invalidate: the server can queue a settings change
rather than applying it, returning it in a `_pending` block that the
form renders as a badge. Writing the local draft into the cache would
show a queued change as applied.

Most mutations are not like that. A "rename a team" write, where the
response is the new team, is a `setQueryData` case.

One gotcha: `mutate` does not throw, `mutateAsync` does. An awaited
`mutateAsync` without a `try/catch` is an unhandled rejection.
2026-09-01 21:58:50 +00:00
Anthony Stirling c57a2a45de Add v2 client-side PDF text editor (#6500)
# Description of Changes

<!--
Please provide a summary of the changes, including:

- What was changed
- Why the change was made
- Any challenges encountered

Closes #(issue_number)
-->

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-09-01 20:55:59 +01:00
ConnorYoh d30faf246b fix(billing): the paid tier is Team, and it is not unlimited users (#7730)
Copy only. No behaviour, no lookup keys, no licence semantics, no
backend.

## Current state

Every surface that sells the paid self-hosted tier offers **"unlimited
seats"** for **"$99/server/mo"**, and the portal's free plan badges
**"Unlimited users"** and **"SSO included"** as free-tier facts.

## Problem

Both claims are now enforceably false.
[#7492](https://github.com/Stirling-Tools/Stirling-PDF/pull/7492) makes
the licence carry a real user cap, and
[Stirling-PDF-SaaS#325](https://github.com/Stirling-Tools/Stirling-PDF-SaaS/pull/325)
sells capacity in blocks of 100 users. An admin reading "unlimited
seats" and then hitting a 409 at the invite screen is the worst version
of this.

The demo has already dropped both claims; ours were the last ones
standing.

## Solution

| Surface | Was | Now |
|---|---|---|
| Onboarding licence slide | "Stirling Server plan, **unlimited seats**
… $99/server/mo" | "Stirling Team plan, **100 users** … $99/mo" |
| Plan comparison table | `unlimitedUsers` = "Unlimited users" |
`usersIncluded` = "100 users included" |
| Plan card highlights | "Unlimited users" | "100 users included" |
| Static plan section | `name: "Server"`, `maxUsers: "Unlimited users"`
| `plan.team.name`, `plan.team.maxUsers` |
| Upgrade banner | "Upgrade to Server Plan" / "unlimited users" |
"Upgrade to the Team plan" / "100 users, SSO" |
| Portal free plan | "Editor" + "SSO included" + "Unlimited users" |
"Editor" + "Every PDF tool" + "Web, desktop & self-hosted" |

The i18n keys are **renamed** (`unlimitedUsers` to `usersIncluded`)
rather than just revalued, so the key name cannot outlive the claim.

Also drops "per server" from `plan.licenseWarning` — we price a block of
100 users and count the provisioned roster, never nodes. And deletes the
orphaned `[settings.planBilling.tier]` block: zero source references,
and it described a retired model (50 credits/mo free, 500 included plus
overage billing).

## Deliberately unchanged

**"Processor" stays the name of the product surface.** The demo names
each plan for its price tier (Editor = $0, Team = $99/mo, Credits = 1¢
each) while keeping Processor as the surface a plan unlocks. Renaming
the surface here would conflate the two, so the plan-name split is left
for the explicit plan catalogue. The free plan also gains no "500 free
credits monthly" badge yet: that is true in the demo but not in our
backend, which still grants a one-time lifetime pool.

## How to test

Self-hosted, as an admin over the free user limit: Settings → Plan
should offer the Team plan at "100 users included", and the onboarding
licence slide should no longer promise unlimited seats. On the portal
billing page, the free plan should read "Free" with no SSO or
unlimited-users badge.

Green locally: 4/4 i18n audits (missing, unused, structure,
translation), 876 tests across 110 files, oxlint, prettier, and all four
typecheck variants (core, proprietary, saas, portal).
2026-09-01 19:09:57 +00:00
EthanHealy01 f6661a8f87 Failure action slots, resolve transition, and the bell that renders them (Review Flow PR 5a) (#7761)
Review Flow PR 5a — the first half of #7479, which stays open for
reference until both halves land. This PR is the ranking and the
bookkeeping; #7762 adds the retry handlers. Merging both reproduces
#7479's diff byte-for-byte.

## What's added

**The action slot model (backend).** `FailureActionSlot` ranks each of a
kind's offers as its `RESOLUTION`, `SECONDARY` or `OVERFLOW`.
`FailureKind` now declares placement per offer — the password-protected
kind names `DECRYPT_AND_RETRY` as its resolution, `UNKNOWN` leads with a
plain `RETRY` — and `FailureActionId` gains those two ids. The
declarations are data; their client handlers arrive in the follow-up, so
this build withholds them with a reason rather than rendering unwired
buttons (the same forward-compatibility #7478 relied on).

**A resolve transition.** `POST /api/v1/notifications/{id}/resolved`
lets a client report a failure fixed. `NotificationSource.parse` turns a
qualified notification id back into the source that owns it, and
`FileRunEventService` folds the resolution into the incident rather than
deleting it.

**`viewerReviewsTeam` on the list response.** A member sees only rows
whose document this browser holds — they can neither open nor fix
anything else — while a team reviewer keeps every row.

**The bell renders the ranking** (`promoteActions`): one primary button,
at most one secondary, the rest in an overflow menu beside **Copy log**.
The row's body is the kind's own sentence; the raw failure message moves
into the menu.

**Read state is a timestamp, not a row id.** `readThroughAt` replaces
`lastSeenId`: when a resolved or dismissed row leaves the list, the rows
below it stay read instead of re-lighting the badge.

## How to test

Needs a proprietary or SaaS build with login enabled (`task dev:all`,
sign in).

1. **Create a failure.** Add a password-protected PDF to the editor and
choose **Skip for now**; the upload's policy run fails on it.
2. **Open the bell.** The row reads the kind's sentence, not a stack
trace. Its primary button is **View file** — the server offers Decrypt
and retry as the resolution, but this build withholds it (handler lands
in the follow-up), so the best renderable offer is promoted instead.
3. **Open the row's ⋯ menu.** View in processor and Dismiss sit there,
along with **Copy log**, which copies the raw message.
4. **Check the read marker survives a departure.** With two failures,
open the bell (badge clears), dismiss the newer row, and refresh: the
badge stays dark. On main, the marker held the departed row's id and the
older row re-read as unread.
5. **Member visibility.** As a plain member, a failure recorded from
another browser does not appear in the bell; as a team reviewer it does.
6. **Resolve endpoint.** `POST
/api/v1/notifications/failure-{eventId}/resolved` as the owner removes
the row on the next poll; `NotificationResolveTest` pins refusal for a
non-owner, an unknown id, and a foreign prefix.

## Migration

None.
2026-09-01 13:25:19 +00:00
Anthony StirlingandJames Brunton ceeec53df4 Let a pipeline run on the editor, on upload or export (#7581)
Redesigns the policies system so that the backend has an understanding
of policies running over the Editor. The Editor is not set up as a
source for the backend because the backend can't actively get files from
it, they come in via the frontend sending them to the backend, so
instead pipelines have a specific editor key in them to encode whether
the pipeline is triggered on file upload/export in the editor.

Also make a big effort in the frontend code towards genericising policy
running. Previously, there was specific support in the main policy
executor for each policy that it had to run, which was not going to be
appropriate long-term, especially when users can run any pipeline in the
editor. There's more work needed here for me to really be happy with it
but this PR is plenty large on its own and moves it in the right
direction.

All of the above was required to allow arbitrary user pipelines to run
in the editor. This PR makes it so that the user can select Editor as a
source in the pipeline creator, along with whether it should run on
upload or export.

<img width="1437" height="506" alt="image"
src="https://github.com/user-attachments/assets/b2d176a1-185c-480b-9916-abdd1447d8e1"
/>

---------

Co-authored-by: James Brunton <james@stirlingpdf.com>
2026-09-01 13:12:06 +00:00
ConnorYoh 4ef2e3811c ci(preview): give PR previews the Stirling account config they need to link (#7728)
Add CI steps to enable PR deploy servers to link to prod saas. This will
allow pr testing of payment flows, usage of real credits etc
2026-09-01 12:35:57 +00:00
ConnorYoh 31d52d4c32 Connect flow for self-hosted account linking, and the triggers that drive it (#7415)
Replaces the bare account-link login box with a guided Connect flow, and
wires up the triggers that actually put it in front of someone.
## Top bar 
<img width="1580" height="422" alt="image"
src="https://github.com/user-attachments/assets/719e12fc-121a-4caa-bc72-124c5167b011"
/>

## The modal

Three steps on the portal's own `FlowModal` + `StepModalHeader`, the
shells procurement and prepay already wear:

1. **What you unlock** — six benefits as a plain list.
<img width="817" height="503" alt="image"
src="https://github.com/user-attachments/assets/4644ddd2-6181-44e1-9be9-7a961972195d"
/>

2. **Sign in** — the existing `SupabaseLoginForm`, reseated.
<img width="880" height="930" alt="image"
src="https://github.com/user-attachments/assets/fc66cbbb-9f98-40a4-9daa-4f2447713f39"
/>

3. **Connected** — confirms, then deep links into Users, Pipelines and
Policies.
<img width="876" height="752" alt="image"
src="https://github.com/user-attachments/assets/28358e4d-a44f-4118-a8ae-8275984ebd00"
/>


Re-auth stays a single step with no pitch and no success screen.

## The triggers

**`LinkGate` stops being dead code.** It was built as the drop-anywhere
"link to unlock" wrapper and was imported by nothing. It is now a
blocking empty state that replaces the feature it guards, wired into
Pipelines, Policies, Users, Sources and Integrations.

**Scoped to creating and editing, never viewing.** Existing pipelines,
policies, sources and connections keep listing and running, so upgrading
an unlinked instance cannot take away something that already works. The
clicks that would open a builder or a create modal ask for the
connection first, which is the moment an admin has already declared
intent.

## Capability signal

`accountLinkAvailable` on `/api/v1/config/app-config`. Gating needs two
facts: whether the instance is linked (`LinkContext`) and whether it
*could* be (this flag). The account-link endpoints 404 when the feature
flag is off, which the client cannot distinguish from "not linked yet" —
so gating on link state alone would lock all five views on every default
install with no way out. `useConnectGate` holds that decision in one
place and shares the app-config query key, so it costs no extra request.

Read from the environment rather than `AccountLinkProperties` because
`:core` cannot depend on `:proprietary`.
2026-09-01 10:39:57 +00:00
627 changed files with 78292 additions and 12452 deletions
+46
View File
@@ -220,6 +220,42 @@ jobs:
echo "app_short=${APP_HASH:0:8}" >> $GITHUB_OUTPUT
fi
# The Stirling account previews connect to. Derived from the ref rather than stored as a URL
# so it cannot drift from the key: a mismatched pair is accepted by the browser and rejected
# by Supabase, surfacing much later as "session expired" on Usage rather than at sign-in.
# Secret only to match Saas-Dev-Deploy.yml, which owns the same value; a project ref is not
# itself sensitive, which is why SAAS_API_BASE_URL next to it is a plain variable.
- name: Resolve Stirling account config
id: saas
env:
PROJECT_REF: ${{ secrets.SAAS_DB_PROJECT_REF }}
API_BASE_OVERRIDE: ${{ vars.SAAS_API_BASE_URL }}
run: |
# Set, this is the one value both halves use: the browser's portal reads and the backend's
# register/entitlement calls have to land on the same SaaS, and nothing checks that they
# do. Unset, only the backend gets a base, from its own compiled-in default.
API_BASE="${API_BASE_OVERRIDE:-https://stirling.com/app}"
echo "backend_base=${API_BASE}" >> "$GITHUB_OUTPUT"
if [ -z "${PROJECT_REF}" ]; then
echo "Not configured for this environment: the preview will build without a Stirling"
echo "account, and the connect dialog will say so. To wire one up, set on the"
echo "pr-preview environment the secrets SAAS_DB_PROJECT_REF and"
echo "SAAS_SUPABASE_PUBLISHABLE_KEY, both from the same Supabase project."
echo "supabase_url=" >> "$GITHUB_OUTPUT"
echo "frontend_base=" >> "$GITHUB_OUTPUT"
else
# Only whether, not which: the ref is a secret here, so Actions masks it out of any
# line it appears in, derived URL included.
echo "Stirling account configured, at ${API_BASE}."
echo "supabase_url=https://${PROJECT_REF}.supabase.co" >> "$GITHUB_OUTPUT"
# Deliberately the override and not API_BASE: the backend's default is a subpath URL
# nobody has confirmed answers /api/v1, and prod CORS does not list preview hostnames,
# so portal reads stay off until someone sets a base they have checked. Empty leaves the
# committed .env default alone, which is the clean "not configured" state.
echo "frontend_base=${API_BASE_OVERRIDE}" >> "$GITHUB_OUTPUT"
fi
- name: Check if image exists
id: check-image
run: |
@@ -246,6 +282,9 @@ jobs:
build-args: |
VERSION_TAG=v2-alpha
BUILD_PORTAL=${{ env.BUILD_PORTAL }}
VITE_SUPABASE_URL=${{ steps.saas.outputs.supabase_url }}
VITE_SUPABASE_PUBLISHABLE_DEFAULT_KEY=${{ secrets.SAAS_SUPABASE_PUBLISHABLE_KEY }}
VITE_SAAS_API_URL=${{ steps.saas.outputs.frontend_base }}
platforms: linux/amd64
- name: Set up SSH
@@ -279,6 +318,13 @@ jobs:
environment:
DISABLE_ADDITIONAL_FEATURES: "false"
STIRLING_BILLING_ACCOUNT_LINK_ENABLED: "true"
STIRLING_BILLING_ACCOUNT_LINK_SAAS_BASE_URL: "${{ steps.saas.outputs.backend_base }}"
# Off so preview traffic never accrues against a real wallet or trips its cap. The
# 402 gate is separate and stays on, so gating is still testable here.
STIRLING_BILLING_ACCOUNT_LINK_METERING_ENABLED: "false"
# Stated rather than inferred from the request: the callback has to come back to the
# preview hostname, not to the container's own :8080 behind this proxy.
SYSTEM_FRONTENDURL: "https://${V2_PORT}.ssl.stirlingpdf.cloud"
SECURITY_ENABLELOGIN: "true"
SECURITY_INITIALLOGIN_USERNAME: "${TEST_LOGIN_USERNAME}"
SECURITY_INITIALLOGIN_PASSWORD: "${TEST_LOGIN_PASSWORD}"
+1
View File
@@ -312,3 +312,4 @@ docs/type3/signatures/
# Local screenshot artifacts from *-screenshots.spec.ts
frontend/editor/screenshots/
frontend/editor/src-tauri/libs/.variant
+3
View File
@@ -306,6 +306,9 @@ tasks.register('copyFrontendAssets', Copy) {
// Exclude files that conflict with backend static resources
exclude 'robots.txt' // Backend already has this
exclude 'favicon.ico' // Backend already has this
// Backend ships its own NotoSans-Regular.ttf here and it is git-tracked;
// letting the editor's copy win would dirty the source tree on every build.
exclude 'fonts/NotoSans-Regular.ttf'
}
into resourcesStaticDir
duplicatesStrategy = DuplicatesStrategy.INCLUDE // Let frontend overwrite when needed
@@ -0,0 +1,598 @@
package stirling.software.SPDF.controller.api;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Base64;
import java.util.List;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDPage;
import org.apache.pdfbox.pdmodel.PDResources;
import org.apache.pdfbox.pdmodel.font.PDFont;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import com.fasterxml.jackson.annotation.JsonInclude;
import io.swagger.v3.oas.annotations.Operation;
import lombok.Data;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.annotations.api.GeneralApi;
import stirling.software.common.service.CustomPDFDocumentFactory;
/**
* Charcode-encode helper for the v2 PDF text editor.
*
* <p>The frontend editor uses PDFium-WASM, which exposes {@code FPDFText_SetCharcodes} for writing
* new text using raw font charcodes (skipping PDFium's broken reverse Unicode→CID lookup for
* embedded subset fonts). What PDFium does NOT expose is the byte-encoding side of an existing font
* - given a PDFont and a Unicode string, what are the bytes the font's encoding produces? PDFBox
* does have that ({@link PDFont#encode}).
*
* <p>This endpoint accepts the source PDF + a "locator" describing where to find the font in
* question (page index + a sample char known to render in the target font, optionally narrowed by
* the font's /BaseFont name) + the Unicode text the frontend wants to encode. It returns the
* charcode sequence the frontend can pass to {@code FPDFText_SetCharcodes}.
*
* <p>If the locator can't find a matching text fragment, or if the font can't encode some chars,
* the response reports which chars are missing so the frontend can fall back to Helvetica per char.
*/
@Slf4j
@GeneralApi
@RequiredArgsConstructor
public class PdfTextEditorCharcodeController {
/** Reject JSON bodies whose base64 implies a decoded PDF larger than this. */
private static final int MAX_PDF_BYTES = 100 * 1024 * 1024;
/**
* Upper bound on {@code request.text} code units. Editor requests are word-sized; an unbounded
* text drove a per-code-point encode/exception loop (CPU burn) on crafted requests.
*/
private static final int MAX_TEXT_CHARS = 4096;
/** Nested form-XObject resource dictionaries visited per lookup (cycle/DoS guard). */
private static final int MAX_RESOURCE_DICTS = 32;
/** Bound on the reverse-map cache so a busy multi-document server can't grow it forever. */
private static final int REVERSE_MAP_CACHE_MAX = 32;
/** Access-ordered LRU bounded at {@link #REVERSE_MAP_CACHE_MAX} entries. */
private static final class BoundedReverseMapCache
extends java.util.LinkedHashMap<String, java.util.Map<String, Long>> {
private static final long serialVersionUID = 1L;
BoundedReverseMapCache() {
super(16, 0.75f, true);
}
@Override
protected boolean removeEldestEntry(
java.util.Map.Entry<String, java.util.Map<String, Long>> eldest) {
return size() > REVERSE_MAP_CACHE_MAX;
}
}
private static final java.util.Map<String, java.util.Map<String, Long>> REVERSE_MAP_CACHE =
java.util.Collections.synchronizedMap(new BoundedReverseMapCache());
private final CustomPDFDocumentFactory pdfDocumentFactory;
// NOTE: PDFBox's PDSimpleFont emits one "No Unicode mapping for .notdef" WARN per probed
// charcode when buildReverseUnicodeMap iterates 0..0xFFFF, which once flooded info.log to
// ~1.4 GB overnight. That logger is silenced DECLARATIVELY in logback.xml (a config entry ops
// can see and revert) rather than by mutating the global logger from a static block here -
// mutating it at class-load time hid the same warnings from every other tool in the JVM with
// no trace in configuration.
@Data
public static class EncodeCharcodesRequest {
/** Base64-encoded original PDF. The frontend already has the bytes loaded. */
private String pdfBase64;
/** 0-based page index containing the font sample. */
private int pageIndex;
/**
* A char known to exist on the page in the target font. Combined with {@code fontName}
* (when supplied) it locates the source PDFont via its ToUnicode CMap.
*/
private String locatorChar;
/**
* Optional /BaseFont name of the target font (as PDFium's FPDFFont_GetBaseFontName reports
* it). When a page has TWO fonts that both render {@code locatorChar}, this disambiguates
* which one to encode against - otherwise the first font found wins and a cross-font edit
* gets the wrong font's charcode. Null = keep the legacy first-match behaviour.
*/
private String fontName;
/**
* Optional SHA-256 (lowercase hex) of the target font's embedded program bytes (what
* PDFium's FPDFFont_GetFontData returns = the decoded FontFile/FontFile2/FontFile3 stream).
* This is the ONLY unambiguous font identity: PDFium strips the "ABCDEF+" subset tag from
* font names, so every subset of one family reports the same {@code fontName} and a
* name-based lookup can land on a SIBLING subset whose charcode space is different -
* returning valid-but-wrong charcodes that scramble the edited text. When present and a
* font on the page matches, it wins over name matching.
*/
private String fontSha256;
/** Unicode text the frontend wants to encode. */
private String text;
}
@Data
@JsonInclude(JsonInclude.Include.NON_NULL)
public static class EncodeCharcodesResponse {
/**
* Per-char charcode array (one entry per code point in {@code request.text}). When the
* font's encoding produces multi-byte sequences, each char gets the full unsigned int value
* of its bytes packed big-endian (so a 2-byte CID like 0x004D becomes 77).
*/
private List<Long> charcodes;
/** Chars from the request that the font couldn't encode. */
private List<String> missing;
/** Diagnostic note - included so the frontend HUD can show what happened. */
private String note;
/** Set when the request failed entirely (bad pdf bytes, no matching font, etc.). */
private String error;
}
@Operation(
summary = "Encode Unicode → font charcodes for the v2 PDF text editor",
description =
"""
Frontend-only helper: takes the source PDF, a locator pointing at an existing
char rendered in the target font, and a Unicode string. Returns the byte
sequence the target font produces for that Unicode, packed as one unsigned
int per char. The frontend then calls FPDFText_SetCharcodes with the
returned ints to inject new text that reuses the embedded font's actual
glyphs. Chars the font can't encode are listed in `missing` so the caller
can fall back per-char.
""")
@PostMapping(
value = "/pdf-text-editor/encode-charcodes",
consumes = "application/json",
produces = "application/json")
public ResponseEntity<EncodeCharcodesResponse> encodeCharcodes(
@RequestBody EncodeCharcodesRequest request) {
EncodeCharcodesResponse resp = new EncodeCharcodesResponse();
if (request == null
|| request.getPdfBase64() == null
|| request.getText() == null
|| request.getLocatorChar() == null) {
resp.setError("missing required fields");
return ResponseEntity.badRequest().body(resp);
}
// length/4*3 bounds the decoded size without decoding, so we reject early before
// allocating.
String b64 = request.getPdfBase64();
if ((long) b64.length() / 4 * 3 > MAX_PDF_BYTES) {
resp.setError("pdf too large");
return ResponseEntity.status(413).body(resp);
}
// Reported separately: a combined check names only one cause and misleads the caller.
if (request.getText().length() > MAX_TEXT_CHARS) {
resp.setError("text too long");
return ResponseEntity.badRequest().body(resp);
}
if (request.getLocatorChar().length() > 4) {
resp.setError("locatorChar too long");
return ResponseEntity.badRequest().body(resp);
}
byte[] pdfBytes;
try {
pdfBytes = Base64.getDecoder().decode(b64);
} catch (IllegalArgumentException e) {
resp.setError("pdfBase64 is not valid base64");
return ResponseEntity.badRequest().body(resp);
}
try (PDDocument doc = pdfDocumentFactory.load(pdfBytes, true)) {
if (request.getPageIndex() < 0 || request.getPageIndex() >= doc.getNumberOfPages()) {
resp.setError("pageIndex out of range");
return ResponseEntity.badRequest().body(resp);
}
PDPage page = doc.getPage(request.getPageIndex());
// Skip walking the page's content stream (it crashes on Type3 fonts with
// UnsupportedOperationException("Not implemented: Type3") before we can do anything
// useful). Instead enumerate the page's font resources and pick the one identified by
// the request's font-program hash (definitive), falling back to name matching.
// For Chrome/Skia-printed PDFs that emit one Type3 font per glyph, this lands on
// the exact font that renders the locator char.
ResourceFont located =
findFontByToUnicode(
page,
request.getLocatorChar(),
request.getFontName(),
request.getFontSha256(),
doc);
if (located == null) {
resp.setError(
"no font on page "
+ request.getPageIndex()
+ " renders locatorChar="
+ request.getLocatorChar()
+ (request.getFontName() != null
? " (fontName=" + request.getFontName() + ")"
: ""));
return ResponseEntity.ok(resp);
}
// Build a reverse Unicode→charcode map by walking the font's ToUnicode CMap.
// This is the ONLY path that works for Type3 fonts (PDFBox's font.encode() throws
// "Not implemented: Type3" on them), and it also acts as a more reliable fallback
// for subset fonts whose encode() rejects chars not in the original document.
//
// For Sample.pdf specifically, every embedded font is Type3 (Chrome/Skia output),
// but they all carry a ToUnicode CMap mapping CIDs back to Unicode. We iterate
// charcodes 0..0xFFFF, call font.toUnicode(cc) for each, and record the inverse
// mapping for the chars the user wants to write.
PDFont font = located.font();
java.util.Map<String, Long> reverseMap =
buildReverseUnicodeMap(pdfBytes, located, request.getPageIndex());
List<Long> charcodes = new ArrayList<>();
List<String> missing = new ArrayList<>();
String text = request.getText();
int i = 0;
while (i < text.length()) {
int cp = text.codePointAt(i);
String oneChar = new String(Character.toChars(cp));
i += Character.charCount(cp);
// Whitespace is NEVER charcode-reused. Subset Type1/LaTeX fonts
// usually have no real space glyph, yet font.encode(0x20) still
// returns code 0x20 without throwing - and SetCharcodes(0x20)
// then paints whatever glyph sits at that subset code (e.g. „
// quotedblbase in LMRoman). Report whitespace as missing so the
// frontend emits it as a positional gap instead.
if (Character.isWhitespace(cp)) {
missing.add(oneChar);
continue;
}
// 1st try: font.encode() - works for Type0/TrueType/Type1
Long packed = null;
try {
byte[] encoded = font.encode(oneChar);
long p = 0L;
for (byte b : encoded) p = (p << 8) | (b & 0xff);
packed = p;
} catch (IOException
| IllegalArgumentException
| UnsupportedOperationException encodeEx) {
// 2nd try: ToUnicode reverse lookup - works for Type3 + anything with a CMap
packed = reverseMap.get(oneChar);
}
if (packed != null) charcodes.add(packed);
else missing.add(oneChar);
}
resp.setCharcodes(charcodes);
if (!missing.isEmpty()) resp.setMissing(missing);
resp.setNote(
"font="
+ font.getName()
+ " encoded "
+ charcodes.size()
+ " of "
+ (charcodes.size() + missing.size())
+ " chars");
return ResponseEntity.ok(resp);
} catch (IOException e) {
log.warn("encodeCharcodes: failed to load PDF", e);
resp.setError("failed to load PDF");
return ResponseEntity.badRequest().body(resp);
} catch (RuntimeException e) {
log.warn("encodeCharcodes: unexpected error", e);
resp.setError("unexpected error");
return ResponseEntity.status(500).body(resp);
}
}
/**
* Locate the font the request targets. Identity sources, strongest first:
*
* <ol>
* <li><b>Program hash</b>: SHA-256 of the embedded font program bytes. Definitive - two
* different subsets NEVER share program bytes, and PDFium's FPDFFont_GetFontData returns
* exactly the decoded FontFile stream, so frontend and backend hash the same bytes.
* <li><b>Exact /BaseFont name</b> (subset tag included), then <b>tag-stripped name</b>. Name
* matches are only accepted when UNAMBIGUOUS: PDFium reports subset fonts WITHOUT their
* "ABCDEF+" tag, so a page with several subsets of one family ("AAAAAC+Garamond",
* "AAAAAG+Garamond", ...) has them ALL match the stripped name - and encoding against the
* wrong sibling returns valid-but-wrong charcodes that scramble the edited text ("RUSSELL
* W. MANGUM" rendered "US EEL W. MANGS M"). With 2+ candidates we return null so the
* frontend takes its safe fallback instead of a coin flip.
* </ol>
*
* <p>This avoids running PDFStreamEngine.processPage, which throws
* UnsupportedOperationException on Type3 font glyph rendering. The PDFont lookup itself is
* purely metadata-driven and works on all subtypes.
*/
private static ResourceFont findFontByToUnicode(
PDPage page, String wantChar, String fontName, String fontSha256, PDDocument doc) {
try {
List<ResourceFont> fonts = collectResourceTreeFonts(page.getResources());
// 1) Program-hash identity. When several dicts share one program (identical bytes
// re-embedded), any of them renders the same glyphs for the same codes; prefer the
// one whose ToUnicode covers the locator char so the reverse map is usable.
if (fontSha256 != null && !fontSha256.isEmpty()) {
List<ResourceFont> hashMatches = new ArrayList<>();
for (ResourceFont rf : fonts) {
String sha = fontProgramSha256(rf.font());
if (fontSha256.equalsIgnoreCase(sha)) hashMatches.add(rf);
}
for (ResourceFont rf : hashMatches) {
if (probesToUnicode(rf.font(), wantChar)) return rf;
}
if (!hashMatches.isEmpty()) return hashMatches.get(0);
// No program on this page hashes to what the frontend is editing (e.g. PDFium
// returned a substitute font's bytes for a non-embedded font). Fall through to
// name matching rather than failing outright.
}
// 2) Name identity - exact tag-included first, then tag-stripped - each accepted
// only when it selects a single font.
if (fontName != null && !fontName.isEmpty()) {
ResourceFont exact =
selectUnambiguous(
fonts, wantChar, f -> fontName.equals(f.getName()), "exact");
if (exact != null) return exact;
String wantStripped = stripSubsetTag(fontName);
ResourceFont stripped =
selectUnambiguous(
fonts,
wantChar,
f -> wantStripped.equals(stripSubsetTag(f.getName())),
"stripped");
if (stripped != null) return stripped;
// The frontend NAMED the font it is editing. Falling back to "any font that
// renders the char" would hand back a DIFFERENT font's charcodes, which the
// frontend then writes into the named font's text object - wrong glyph, and the
// backend strategy skips all frontend validation. Report the char missing
// instead so the caller takes its own fallback path.
return null;
}
// 3) Legacy locator-only behaviour: first font whose ToUnicode renders the char.
for (ResourceFont rf : fonts) {
if (probesToUnicode(rf.font(), wantChar)) return rf;
}
} catch (RuntimeException ignore) {
// Be defensive: any single bad font shouldn't sink the whole request.
}
return null;
}
/**
* Apply {@code nameFilter}, then decide: exactly one candidate whose ToUnicode covers {@code
* wantChar} wins; two+ probe-hits are AMBIGUOUS (null). With zero probe-hits, a single
* name-matching font is still returned (font.encode() may handle chars without a ToUnicode -
* common for Type0/Identity-H), but two+ name matches are again ambiguous.
*/
private static ResourceFont selectUnambiguous(
List<ResourceFont> fonts,
String wantChar,
java.util.function.Predicate<PDFont> nameFilter,
String modeLabel) {
List<ResourceFont> named = new ArrayList<>();
for (ResourceFont rf : fonts) {
try {
if (rf.font().getName() != null && nameFilter.test(rf.font())) named.add(rf);
} catch (RuntimeException ignore) {
}
}
if (named.isEmpty()) return null;
List<ResourceFont> probed = new ArrayList<>();
for (ResourceFont rf : named) {
if (probesToUnicode(rf.font(), wantChar)) probed.add(rf);
}
if (probed.size() == 1) return probed.get(0);
if (probed.size() > 1) {
log.debug(
"encodeCharcodes: {} name match ambiguous ({} fonts render locator '{}') -"
+ " refusing cross-subset guess",
modeLabel,
probed.size(),
wantChar);
return null;
}
return named.size() == 1 ? named.get(0) : null;
}
/** True when some charcode in the font's ToUnicode CMap maps to {@code wantChar}. */
private static boolean probesToUnicode(PDFont font, String wantChar) {
// Cheap inverse-CMap probe: iterate codes until we hit one whose toUnicode is wantChar.
// For Type3 with at most ~16 glyphs, this is microseconds. For full Type0 subsets
// it's a few-thousand-iteration scan.
int upper = font.isStandard14() ? 256 : 0x10000;
for (int cc = 0; cc < upper; cc++) {
String u;
try {
u = font.toUnicode(cc);
} catch (Exception ignore) {
continue;
}
if (u != null && u.equals(wantChar)) return true;
}
return false;
}
private record ResourceFont(PDFont font, String path) {}
private record PendingResources(PDResources resources, String path) {}
/**
* Breadth-first collection of every distinct font reachable from the page's resources AND every
* nested form XObject's resources (bounded by {@link #MAX_RESOURCE_DICTS}, cycle-safe, deduped
* by COS dictionary identity). The v2 reader surfaces form-XObject text as editable, so its
* fonts must be findable too.
*/
private static List<ResourceFont> collectResourceTreeFonts(PDResources resources) {
List<ResourceFont> out = new ArrayList<>();
java.util.ArrayDeque<PendingResources> queue = new java.util.ArrayDeque<>();
java.util.Set<org.apache.pdfbox.cos.COSDictionary> seenDicts =
java.util.Collections.newSetFromMap(new java.util.IdentityHashMap<>());
java.util.Set<org.apache.pdfbox.cos.COSDictionary> seenFonts =
java.util.Collections.newSetFromMap(new java.util.IdentityHashMap<>());
if (resources != null) queue.add(new PendingResources(resources, ""));
int visited = 0;
// Bound a crafted page declaring many fonts none of which match (CPU-DoS guard).
final int MAX_FONTS = 64;
while (!queue.isEmpty() && visited < MAX_RESOURCE_DICTS) {
PendingResources pending = queue.poll();
PDResources res = pending.resources();
if (!seenDicts.add(res.getCOSObject())) continue;
visited++;
for (org.apache.pdfbox.cos.COSName name : res.getFontNames()) {
if (out.size() >= MAX_FONTS) break;
PDFont font;
try {
font = res.getFont(name);
} catch (IOException | RuntimeException e) {
continue;
}
if (font == null || !seenFonts.add(font.getCOSObject())) continue;
out.add(new ResourceFont(font, pending.path() + "/" + name.getName()));
}
try {
for (org.apache.pdfbox.cos.COSName xn : res.getXObjectNames()) {
try {
org.apache.pdfbox.pdmodel.graphics.PDXObject xo = res.getXObject(xn);
if (xo
instanceof
org.apache.pdfbox.pdmodel.graphics.form.PDFormXObject form) {
PDResources fr = form.getResources();
if (fr != null) {
queue.add(
new PendingResources(
fr, pending.path() + "/" + xn.getName()));
}
}
} catch (IOException | RuntimeException ignore) {
}
}
} catch (RuntimeException ignore) {
}
}
return out;
}
/**
* SHA-256 (lowercase hex) of a font's embedded program bytes - the decoded
* FontFile/FontFile2/FontFile3 stream, which is byte-identical to what PDFium's
* FPDFFont_GetFontData hands the frontend. Null when the font embeds no program.
*/
private static String fontProgramSha256(PDFont font) {
try {
org.apache.pdfbox.pdmodel.font.PDFontDescriptor fd = font.getFontDescriptor();
if (fd == null && font instanceof org.apache.pdfbox.pdmodel.font.PDType0Font type0) {
fd = type0.getDescendantFont().getFontDescriptor();
}
if (fd == null) return null;
org.apache.pdfbox.pdmodel.common.PDStream stream = fd.getFontFile2();
if (stream == null) stream = fd.getFontFile3();
if (stream == null) stream = fd.getFontFile();
if (stream == null) return null;
return sha256Hex(stream.toByteArray());
} catch (IOException | RuntimeException e) {
return null;
}
}
/** Drop the 6-letter "ABCDEF+" subset prefix PDF puts on subset /BaseFont names. */
private static String stripSubsetTag(String fontName) {
if (fontName == null) return null;
if (fontName.length() > 7
&& fontName.charAt(6) == '+'
&& fontName.chars().limit(6).allMatch(c -> c >= 'A' && c <= 'Z')) {
return fontName.substring(7);
}
return fontName;
}
/**
* Build a Unicode→charcode map for a font by iterating every charcode in 0..0xFFFF and asking
* the font's ToUnicode CMap what Unicode it maps to. Charcodes that aren't in the CMap throw
* inside toUnicode (PDFBox returns null or throws depending on font subtype), and those are
* skipped silently.
*
* <p>This is the encoding inverse PDFBox doesn't expose directly. For Type3 fonts (where
* font.encode() throws "Not implemented"), this is the ONLY way to write text in the same font
* - we look up the user's char in the reverse map and pass that charcode to
* FPDFText_SetCharcodes on the frontend.
*
* <p>The 0..0xFFFF range is sufficient for Type0/CIDFontType2 fonts (CIDs are 16-bit). For
* single-byte fonts the loop short-circuits after 256. We don't go higher because no PDF font
* has a CID outside that range in practice; the per-font result is memoised in {@link
* #REVERSE_MAP_CACHE} so the 65 536-entry probe runs once per document+font, not per request.
*/
private static java.util.Map<String, Long> buildReverseUnicodeMap(
byte[] pdfBytes, ResourceFont located, int pageIndex) {
String key = sha256Hex(pdfBytes) + "|" + fontCacheIdentity(located, pageIndex);
// Compound get/put under the map's own monitor. The 0..0xFFFF probe runs OUTSIDE the
// lock so one slow build can't block every other request on the shared cache.
java.util.Map<String, Long> cached;
synchronized (REVERSE_MAP_CACHE) {
cached = REVERSE_MAP_CACHE.get(key);
}
if (cached != null) return cached;
java.util.Map<String, Long> built = computeReverseUnicodeMap(located.font());
synchronized (REVERSE_MAP_CACHE) {
java.util.Map<String, Long> raced = REVERSE_MAP_CACHE.putIfAbsent(key, built);
return raced != null ? raced : built;
}
}
private static String fontCacheIdentity(ResourceFont located, int pageIndex) {
org.apache.pdfbox.cos.COSObjectKey objectKey = null;
try {
objectKey = located.font().getCOSObject().getKey();
} catch (RuntimeException ignore) {
}
if (objectKey != null) {
return "obj|" + objectKey.getNumber() + "." + objectKey.getGeneration();
}
return "res|p" + pageIndex + located.path();
}
/** Lowercase hex SHA-256 of the PDF bytes; used as the reverse-map cache key. */
private static String sha256Hex(byte[] bytes) {
try {
byte[] digest = java.security.MessageDigest.getInstance("SHA-256").digest(bytes);
StringBuilder sb = new StringBuilder(digest.length * 2);
for (byte b : digest) {
sb.append(Character.forDigit((b >> 4) & 0xf, 16));
sb.append(Character.forDigit(b & 0xf, 16));
}
return sb.toString();
} catch (java.security.NoSuchAlgorithmException e) {
// SHA-256 is always present in a JRE; fall back to a length+hash key just in case so
// the cache still functions (correctness holds - collisions only cost a rebuild).
return bytes.length + ":" + java.util.Arrays.hashCode(bytes);
}
}
private static java.util.Map<String, Long> computeReverseUnicodeMap(PDFont font) {
java.util.Map<String, Long> out = new java.util.HashMap<>();
int upper = font.isStandard14() ? 256 : 0x10000;
for (int cc = 0; cc < upper; cc++) {
String u;
try {
u = font.toUnicode(cc);
} catch (Exception ignore) {
continue;
}
if (u == null || u.isEmpty()) continue;
// First charcode wins for a given Unicode (the canonical mapping).
out.putIfAbsent(u, (long) cc);
}
return out;
}
}
@@ -338,6 +338,19 @@ public class ConfigController {
// Premium/Enterprise settings
configData.put("premiumEnabled", applicationProperties.getPremium().isEnabled());
// Whether this instance can link a Stirling (SaaS) account at all. The account-link
// beans live in :proprietary and are @ConditionalOnProperty on this same key, so when
// it is off they are absent and /api/v1/account-link/* returns 404. The frontend cannot
// tell that 404 apart from "not linked yet", so it needs this told to it explicitly
// before it can prompt anyone to link. Read from the environment rather than
// AccountLinkProperties because :core must not depend on :proprietary.
configData.put(
"accountLinkAvailable",
applicationContext
.getEnvironment()
.getProperty(
"stirling.billing.account-link.enabled", Boolean.class, false));
// AI Engine settings
ApplicationProperties.AiEngine aiEngineConfig = applicationProperties.getAiEngine();
configData.put("aiEngineEnabled", aiEngineConfig.isEnabled());
@@ -154,7 +154,8 @@ public class PdfJsonFontService {
return "otf";
}
if (signature == 0x74746366) {
return "cff";
log.debug("[FONT-DEBUG] TrueType Collection ('ttcf') font program is unsupported");
return null;
}
return null;
}
@@ -175,7 +176,8 @@ public class PdfJsonFontService {
return "otf";
}
if (signature == 0x74746366) {
return "cff";
log.debug("[FONT-DEBUG] TrueType Collection ('ttcf') FontFile2 is unsupported");
return null;
}
return null;
}
+42 -5
View File
@@ -15,26 +15,63 @@
<encoder>
<pattern>%d %p %c{1} [%thread] %m%n</pattern>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${LOG_PATH}/auth-%d{yyyy-MM-dd}.log.gz</fileNamePattern>
<!-- SizeAndTime, not Time alone: the size trigger is what stops a
runaway logger filling the disk (see GENERAL appender note).
Archives are gzipped, so 64 MB of them holds far more than a
day. Worst case on disk is one 100 MB live file plus the cap. -->
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<fileNamePattern>${LOG_PATH}/auth-%d{yyyy-MM-dd}.%i.log.gz</fileNamePattern>
<maxFileSize>100MB</maxFileSize>
<maxHistory>7</maxHistory>
<totalSizeCap>64MB</totalSizeCap>
</rollingPolicy>
</appender>
<!-- Rolling File Appender for General Logs -->
<!-- Rolling File Appender for General Logs
Why SizeAndTimeBased + totalSizeCap: a previous build of the v2 PDF
text editor's reverse-CMap probe loop triggered PDSimpleFont to emit
one "No Unicode mapping for .notdef" WARN per probed charcode per
font per request. With TimeBasedRollingPolicy alone there was no
size ceiling; info.log grew to 1.4 GB in a single day before the JVM
choked. The class-level silencer fixes the specific offender, but
this size cap is the defence-in-depth: any future logger that
floods unexpectedly will roll + auto-delete instead of starving
disk + Jetty threads. -->
<appender name="GENERAL" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${LOG_PATH}/info.log</file>
<encoder>
<pattern>%d %p %c{1} [%thread] %m%n</pattern>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${LOG_PATH}/info-%d{yyyy-MM-dd}.log.gz</fileNamePattern>
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<fileNamePattern>${LOG_PATH}/info-%d{yyyy-MM-dd}.%i.log.gz</fileNamePattern>
<maxFileSize>100MB</maxFileSize>
<maxHistory>7</maxHistory>
<totalSizeCap>256MB</totalSizeCap>
</rollingPolicy>
</appender>
<!-- Suppress PDFBox PDSimpleFont's per-charcode .notdef WARN.
Required by the v2 PDF text editor's `buildReverseUnicodeMap`
which DELIBERATELY iterates every charcode in 0..0xFFFF to
discover the encoding-to-Unicode map of an embedded subset
font. For any subset font ~99% of those probes hit .notdef,
and the default WARN level for those misses turned info.log
into a 1.4 GB monster overnight.
This declarative logback entry is the SOLE mechanism: it is
visible to ops and revertable via configuration. An earlier
build also mutated this logger's level from a static block in
PdfTextEditorCharcodeController, which silenced the same
warnings JVM-wide with no trace in any config file - that
static block has been removed in favour of this entry. -->
<logger name="org.apache.pdfbox.pdmodel.font.PDSimpleFont"
level="ERROR" additivity="false">
<appender-ref ref="CONSOLE"/>
<appender-ref ref="GENERAL"/>
</logger>
<!-- Root Logger -->
<root level="INFO">
<appender-ref ref="CONSOLE"/>
@@ -57,6 +57,8 @@ class ToolIODeclarationCoverageTest {
// documents.
"/api/v1/convert/pdf/text-editor",
"/api/v1/convert/text-editor/pdf",
// Charcode lookup for the v2 editor: returns glyph mappings, not a document.
"/api/v1/general/pdf-text-editor",
// Signing sessions, certificate checks and hardware token enumeration; the
// signing tool itself is /api/v1/security/cert-sign, which is declared.
"/api/v1/security/cert-sign/sessions",
@@ -0,0 +1,516 @@
package stirling.software.SPDF.controller.api;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
import javax.imageio.ImageIO;
import org.apache.pdfbox.Loader;
import org.apache.pdfbox.cos.COSName;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDPage;
import org.apache.pdfbox.pdmodel.PDPageContentStream;
import org.apache.pdfbox.pdmodel.PDResources;
import org.apache.pdfbox.pdmodel.font.PDFont;
import org.apache.pdfbox.pdmodel.font.PDFontDescriptor;
import org.apache.pdfbox.pdmodel.font.PDType0Font;
import org.apache.pdfbox.pdmodel.font.PDType1Font;
import org.apache.pdfbox.pdmodel.font.PDType3Font;
import org.apache.pdfbox.pdmodel.font.Standard14Fonts;
import org.apache.pdfbox.rendering.PDFRenderer;
import org.junit.jupiter.api.Disabled;
import org.junit.jupiter.api.Test;
/**
* Probe: what can PDFBox actually do for font ENCODING on real-world PDFs. This is a diagnostic
* test (not a regression) - run with --tests PdfBoxFontEncodingProbeTest -i to see stdout.
*
* <p>Answers these questions:
*
* <ol>
* <li>Type0/CIDFontType2 subset: can we add a new glyph not in the original subset? (no, encode
* throws IllegalArgumentException).
* <li>Type1: same question.
* <li>TrueType: same question.
* <li>Can we load a fresh TTF via PDType0Font.load(doc, file) and write text with it? (yes,
* primary path).
* <li>Round-trip via getFontStream / re-embed - can it rehabilitate Type3? (no - Type3 has no
* FontFile* program at all).
* <li>What fonts ship with PDFBox / fontbox? (only LiberationSans-Regular.ttf + AFM for the 14
* standard fonts; CFF/Type1 binaries are NOT bundled - Standard14Fonts.getMappedFontName
* redirects unmappable ones to LiberationSans).
* </ol>
*/
@Disabled(
"Diagnostic probe: dumps PDFBox font encoding tables to stdout and asserts nothing. Kept for font debugging; run manually.")
public class PdfBoxFontEncodingProbeTest {
private static final Path PROJECT_ROOT =
Paths.get(System.getProperty("user.dir")).getParent().getParent();
private static final Path SAMPLE =
PROJECT_ROOT.resolve("frontend/editor/public/samples/Sample.pdf");
private static final Path[] EXTRA_FIXTURES = {
PROJECT_ROOT.resolve("frontend/editor/src/core/tests/test-fixtures/stirling-marketing.pdf"),
PROJECT_ROOT.resolve("frontend/editor/src/core/tests/test-fixtures/multi-page-sample.pdf"),
PROJECT_ROOT.resolve("frontend/editor/src/core/tests/test-fixtures/big-sample.pdf"),
PROJECT_ROOT.resolve("frontend/editor/src/core/tests/test-fixtures/paragraph-sample.pdf"),
PROJECT_ROOT.resolve("frontend/editor/src/core/tests/test-fixtures/user-sample.pdf"),
};
/**
* Rasterize the Q4b output (Sample.pdf with injected Liberation text) to confirm the new text
* actually renders on top of the existing Type3 content.
*/
@Test
public void probeRenderInjectedSample() throws IOException {
Path liberation =
PROJECT_ROOT.resolve(
"app/core/src/main/resources/static/fonts/LiberationSans-Regular.ttf");
byte[] pdfBytes = Files.readAllBytes(SAMPLE);
ByteArrayOutputStream out = new ByteArrayOutputStream();
try (PDDocument doc = Loader.loadPDF(pdfBytes)) {
PDPage page = doc.getPage(0);
PDType0Font ttf;
try (InputStream in = Files.newInputStream(liberation)) {
ttf = PDType0Font.load(doc, in, true);
}
try (PDPageContentStream cs =
new PDPageContentStream(
doc, page, PDPageContentStream.AppendMode.APPEND, true, true)) {
cs.beginText();
cs.setFont(ttf, 24);
cs.newLineAtOffset(50, 120);
cs.showText("INJECTED via PDType0Font.load - $@#&Z");
cs.endText();
}
doc.save(out);
}
// Rasterize page 0 to a PNG so we can eyeball it.
try (PDDocument check = Loader.loadPDF(out.toByteArray())) {
PDFRenderer renderer = new PDFRenderer(check);
java.awt.image.BufferedImage img = renderer.renderImageWithDPI(0, 100);
// Build dir, not the repo root: this render is a debugging aid and was
// twice committed by accident when it landed in the working tree.
Path png =
Paths.get(System.getProperty("user.dir"), "build", "probe-output")
.resolve("pdfbox-probe-q4b-rendered.png");
Files.createDirectories(png.getParent());
ImageIO.write(img, "PNG", png.toFile());
System.out.println(
"Rendered injected sample to "
+ png
+ " - "
+ img.getWidth()
+ "x"
+ img.getHeight());
}
}
/**
* Build a PDF in memory that uses a Type0/CIDFontType2 subset font (the kind Word / InDesign /
* LibreOffice produce), then probe whether encode() can add a glyph that wasn't in the original
* subset.
*/
@Test
public void probeType0CIDFontType2Subset() throws IOException {
System.out.println(
"\n##################################################################\n"
+ "Q1 probe: Type0/CIDFontType2 SUBSET can/cannot add new glyphs\n"
+ "##################################################################\n");
Path liberation =
PROJECT_ROOT.resolve(
"app/core/src/main/resources/static/fonts/LiberationSans-Regular.ttf");
// Build a PDF that contains only "abc" subsetted from LiberationSans.
ByteArrayOutputStream baos = new ByteArrayOutputStream();
try (PDDocument doc = new PDDocument()) {
PDPage page = new PDPage();
doc.addPage(page);
PDType0Font subset;
try (InputStream in = Files.newInputStream(liberation)) {
subset = PDType0Font.load(doc, in, true /* embedSubset */);
}
try (PDPageContentStream cs = new PDPageContentStream(doc, page)) {
cs.beginText();
cs.setFont(subset, 12);
cs.newLineAtOffset(100, 700);
cs.showText("abc");
cs.endText();
}
doc.save(baos);
}
// Reload the produced PDF and try to add a NEW glyph through the embedded subset font.
byte[] subsetPdf = baos.toByteArray();
try (PDDocument doc = Loader.loadPDF(subsetPdf)) {
PDResources res = doc.getPage(0).getResources();
for (COSName fn : res.getFontNames()) {
PDFont f = res.getFont(fn);
System.out.println(
" Subset font in saved PDF: "
+ f.getName()
+ " ("
+ f.getClass().getSimpleName()
+ ", subType="
+ f.getSubType()
+ ")");
for (String ch : new String[] {"a", "b", "c", "Z", "z", "0", "$", "@", "X", " "}) {
try {
byte[] enc = f.encode(ch);
StringBuilder hex = new StringBuilder();
for (byte b : enc) hex.append(String.format("%02X ", b & 0xff));
System.out.println(
" encode('" + ch + "') -> [" + hex.toString().trim() + "] OK");
} catch (UnsupportedOperationException uoe) {
System.out.println(" encode('" + ch + "') UNSUPPORTED");
} catch (IllegalArgumentException iae) {
System.out.println(
" encode('" + ch + "') MISSING - " + iae.getMessage());
} catch (IOException ioe) {
System.out.println(" encode('" + ch + "') IO ERR - " + ioe.getMessage());
}
}
}
}
}
@Test
public void probeExtraFixtures() throws IOException {
System.out.println(
"\n##################################################################\n"
+ "Extra fixture font-class probe\n"
+ "##################################################################\n");
for (Path fixture : EXTRA_FIXTURES) {
if (!Files.exists(fixture)) {
System.out.println("(missing) " + fixture);
continue;
}
System.out.println("\n=== " + fixture.getFileName() + " ===");
byte[] bytes = Files.readAllBytes(fixture);
try (PDDocument doc = Loader.loadPDF(bytes)) {
Set<COSName> seen = new HashSet<>();
for (int p = 0; p < doc.getNumberOfPages(); p++) {
PDPage page = doc.getPage(p);
PDResources res = page.getResources();
if (res == null) continue;
for (COSName name : res.getFontNames()) {
if (!seen.add(name)) continue;
try {
PDFont f = res.getFont(name);
if (f == null) continue;
String fontFile = "none";
PDFontDescriptor d = f.getFontDescriptor();
if (d != null) {
if (d.getFontFile() != null) fontFile = "FontFile";
else if (d.getFontFile2() != null) fontFile = "FontFile2";
else if (d.getFontFile3() != null) fontFile = "FontFile3";
}
String z = "?";
try {
f.encode("Z");
z = "OK";
} catch (UnsupportedOperationException ex) {
z = "UNSUPPORTED";
} catch (IllegalArgumentException ex) {
z = "MISSING";
} catch (IOException ex) {
z = "IO_ERR";
}
System.out.println(
" page "
+ p
+ " "
+ name.getName()
+ " -> "
+ f.getName()
+ " "
+ f.getClass().getSimpleName()
+ " ("
+ f.getSubType()
+ ", "
+ fontFile
+ ", embed="
+ f.isEmbedded()
+ ") encode('Z')="
+ z);
} catch (IOException e) {
System.out.println(
" page "
+ p
+ " "
+ name.getName()
+ " load failed: "
+ e.getMessage());
}
}
}
}
}
}
@Test
public void probeAllQuestions() throws IOException {
System.out.println(
"\n##################################################################\n"
+ "PDFBox font-encoding probe (Sample.pdf + bundled fallback fonts)\n"
+ "##################################################################\n");
// Discover every font in Sample.pdf so we have a real-world test set.
byte[] pdfBytes = Files.readAllBytes(SAMPLE);
try (PDDocument doc = Loader.loadPDF(pdfBytes)) {
List<PDFont> allFonts = new ArrayList<>();
Set<COSName> seen = new HashSet<>();
for (int p = 0; p < doc.getNumberOfPages(); p++) {
PDPage page = doc.getPage(p);
PDResources res = page.getResources();
if (res == null) continue;
for (COSName name : res.getFontNames()) {
if (!seen.add(name)) continue;
try {
PDFont f = res.getFont(name);
if (f != null) allFonts.add(f);
} catch (Exception e) {
System.out.println(
" (skipped " + name.getName() + " - " + e.getMessage() + ")");
}
}
}
System.out.println(
"Discovered " + allFonts.size() + " unique fonts across Sample.pdf:");
for (PDFont f : allFonts) {
System.out.println(
" - "
+ f.getName()
+ " ("
+ f.getClass().getSimpleName()
+ ", subType="
+ f.getSubType()
+ ", embedded="
+ f.isEmbedded()
+ ")");
}
// Q1/Q2/Q3
// Try encoding a char that is NEVER in Sample.pdf via each font.
// 'Z' is unlikely to be in the subset for most marketing pages.
// Try several candidates to surface what each font can/can't add.
String[] candidates = {"Z", "$", "@", "#", "Q", "&", "A", "0", "M"};
for (PDFont f : allFonts) {
System.out.println("\n=== Encode-probe for font: " + f.getName() + " ===");
for (String ch : candidates) {
try {
byte[] enc = f.encode(ch);
StringBuilder hex = new StringBuilder();
for (byte b : enc) hex.append(String.format("%02X ", b & 0xff));
System.out.println(
" encode('" + ch + "') -> [" + hex.toString().trim() + "] OK");
} catch (UnsupportedOperationException uoe) {
System.out.println(
" encode('" + ch + "') UNSUPPORTED: " + uoe.getMessage());
} catch (IllegalArgumentException iae) {
System.out.println(" encode('" + ch + "') MISSING: " + iae.getMessage());
} catch (IOException ioe) {
System.out.println(" encode('" + ch + "') IO ERR: " + ioe.getMessage());
}
}
}
// Q5
// For each font, see what's in the FontFile* stream - this is what we'd
// have to round-trip through to "rehabilitate" a Type3 font.
System.out.println("\n=== FontFile stream availability (Q5) ===");
for (PDFont f : allFonts) {
String kind = "none";
int size = 0;
PDFontDescriptor d = f.getFontDescriptor();
if (d != null) {
if (d.getFontFile() != null) {
kind = "FontFile (Type1)";
size = streamBytes(d.getFontFile().getCOSObject().createInputStream());
} else if (d.getFontFile2() != null) {
kind = "FontFile2 (TTF)";
size = streamBytes(d.getFontFile2().getCOSObject().createInputStream());
} else if (d.getFontFile3() != null) {
kind = "FontFile3 (CFF/OpenType)";
size = streamBytes(d.getFontFile3().getCOSObject().createInputStream());
}
}
System.out.println(
" "
+ f.getName()
+ " ("
+ f.getClass().getSimpleName()
+ "): "
+ kind
+ " ("
+ size
+ " bytes)");
if (f instanceof PDType3Font) {
System.out.println(
" -> Type3 has CharProc streams, NOT a FontFile binary."
+ " getFontStream() returns null. Round-trip rehab is impossible:");
System.out.println(
" each glyph is a mini content stream, not a glyph outline in a"
+ " standard font format. We'd need to rasterize each CharProc to"
+ " glyph outlines + build a fresh TTF/CFF from scratch.");
}
}
}
// Q4: PDType0Font.load(doc, file) round-trip
System.out.println("\n=== Q4: load fresh TTF and write text to a fresh PDF ===");
Path liberation =
PROJECT_ROOT.resolve(
"app/core/src/main/resources/static/fonts/LiberationSans-Regular.ttf");
if (!Files.exists(liberation)) {
System.out.println(" Liberation TTF not found at " + liberation);
} else {
try (PDDocument out = new PDDocument()) {
PDPage page = new PDPage();
out.addPage(page);
PDType0Font ttf;
try (InputStream in = Files.newInputStream(liberation)) {
ttf = PDType0Font.load(out, in, true /* embedSubset */);
}
System.out.println(
" Loaded TTF -> "
+ ttf.getName()
+ " ("
+ ttf.getClass().getSimpleName()
+ ")");
String testText = "Hello world! 0123 Z $ @";
byte[] encoded = ttf.encode(testText);
System.out.println(
" Encoded "
+ testText.length()
+ " chars -> "
+ encoded.length
+ " bytes (Identity-H = 2 bytes/glyph)");
try (PDPageContentStream cs = new PDPageContentStream(out, page)) {
cs.beginText();
cs.setFont(ttf, 12);
cs.newLineAtOffset(100, 700);
cs.showText(testText);
cs.endText();
}
ByteArrayOutputStream baos = new ByteArrayOutputStream();
out.save(baos);
Path tmp = Files.createTempFile("pdfbox-probe-q4-", ".pdf");
Files.write(tmp, baos.toByteArray());
System.out.println(
" Wrote fresh-TTF PDF to "
+ tmp
+ " ("
+ baos.size()
+ " bytes) - opens cleanly.");
// Re-load to confirm the new font is embedded properly.
try (PDDocument check = Loader.loadPDF(baos.toByteArray())) {
PDResources res = check.getPage(0).getResources();
for (COSName fn : res.getFontNames()) {
PDFont f = res.getFont(fn);
System.out.println(
" embedded font: "
+ f.getName()
+ " ("
+ f.getClass().getSimpleName()
+ ", embedded="
+ f.isEmbedded()
+ ")");
}
}
}
}
// Q4b: load TTF into an EXISTING PDF (Sample.pdf) and append text
System.out.println(
"\n=== Q4b: load TTF into EXISTING Sample.pdf and write text on page 0 ===");
try (PDDocument doc = Loader.loadPDF(pdfBytes)) {
PDPage page = doc.getPage(0);
PDType0Font ttf;
try (InputStream in = Files.newInputStream(liberation)) {
ttf = PDType0Font.load(doc, in, true);
}
// append-mode content stream so we don't disturb existing graphics
try (PDPageContentStream cs =
new PDPageContentStream(
doc,
page,
PDPageContentStream.AppendMode.APPEND,
true /* compress */,
true /* resetContext */)) {
cs.beginText();
cs.setFont(ttf, 12);
cs.newLineAtOffset(50, 50);
cs.showText("Injected via PDType0Font.load - $@#&");
cs.endText();
}
ByteArrayOutputStream baos = new ByteArrayOutputStream();
doc.save(baos);
Path tmp = Files.createTempFile("pdfbox-probe-q4b-", ".pdf");
Files.write(tmp, baos.toByteArray());
System.out.println(
" Wrote injected-text PDF to " + tmp + " (" + baos.size() + " bytes).");
// Verify by re-reading: how many fonts now on page 0?
try (PDDocument check = Loader.loadPDF(baos.toByteArray())) {
PDResources res = check.getPage(0).getResources();
int count = 0;
for (COSName fn : res.getFontNames()) {
PDFont f = res.getFont(fn);
count++;
System.out.println(
" page-0 font: "
+ fn.getName()
+ " -> "
+ f.getName()
+ " ("
+ f.getClass().getSimpleName()
+ ")");
}
System.out.println(" Total fonts on page 0: " + count);
}
}
// Q6: what fonts ship in PDFBox / fontbox
System.out.println("\n=== Q6: bundled fonts (Standard14 redirect probe) ===");
for (Standard14Fonts.FontName fn : Standard14Fonts.FontName.values()) {
PDType1Font f = new PDType1Font(fn);
String mapped = "" + Standard14Fonts.getMappedFontName(fn.getName());
System.out.println(
" Standard14 "
+ fn.getName()
+ " -> mapped='"
+ mapped
+ "' name="
+ f.getName());
}
System.out.println(
" (PDFBox bundles ONLY LiberationSans-Regular.ttf as a binary; the AFMs cover"
+ " metrics for the 14 standard fonts but rendering Helvetica/Times/Courier"
+ " glyphs falls back to LiberationSans glyphs at runtime when no system font"
+ " is found.)");
}
private static int streamBytes(InputStream is) {
try (InputStream it = is) {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] buf = new byte[4096];
int n;
while ((n = it.read(buf)) >= 0) baos.write(buf, 0, n);
return baos.size();
} catch (IOException e) {
return -1;
}
}
}
@@ -0,0 +1,755 @@
package stirling.software.SPDF.controller.api;
import static org.assertj.core.api.Assertions.assertThat;
import static org.mockito.Mockito.mock;
import java.io.ByteArrayOutputStream;
import java.io.InputStream;
import java.util.Base64;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDPage;
import org.apache.pdfbox.pdmodel.PDPageContentStream;
import org.apache.pdfbox.pdmodel.font.PDType1Font;
import org.apache.pdfbox.pdmodel.font.Standard14Fonts;
import org.junit.jupiter.api.Test;
import org.springframework.http.ResponseEntity;
import stirling.software.SPDF.controller.api.PdfTextEditorCharcodeController.EncodeCharcodesRequest;
import stirling.software.SPDF.controller.api.PdfTextEditorCharcodeController.EncodeCharcodesResponse;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.service.PdfMetadataService;
/**
* Regression coverage for the v2 text editor "spaces render as „" bug.
*
* <p>mushroom-life.pdf is a LaTeX document whose embedded LMRoman subset font has NO real space
* glyph, yet {@code font.encode(" ")} still returns charcode 0x20 without throwing. Reusing that
* code via {@code FPDFText_SetCharcodes} paints whatever glyph sits at subset code 0x20 - the
* quotedblbase „. The controller must therefore report whitespace as {@code missing} so the
* frontend emits it as a positional gap instead of a reused glyph.
*/
class PdfTextEditorCharcodeControllerTest {
private static PdfTextEditorCharcodeController controller() {
return new PdfTextEditorCharcodeController(
new CustomPDFDocumentFactory(mock(PdfMetadataService.class)));
}
private static String mushroomBase64() throws Exception {
try (InputStream in =
PdfTextEditorCharcodeControllerTest.class.getResourceAsStream(
"/pdftexteditor/mushroom-life.pdf")) {
assertThat(in).as("mushroom-life.pdf test resource").isNotNull();
return Base64.getEncoder().encodeToString(in.readAllBytes());
}
}
private static EncodeCharcodesRequest request(String text) throws Exception {
EncodeCharcodesRequest req = new EncodeCharcodesRequest();
req.setPdfBase64(mushroomBase64());
req.setPageIndex(0);
// findFontByToUnicode locates the font via the ToUnicode CMap - "M" exists on page 0.
req.setLocatorChar("M");
req.setText(text);
return req;
}
@Test
void spaceIsReportedMissingNeverEncoded() throws Exception {
PdfTextEditorCharcodeController controller = controller();
ResponseEntity<EncodeCharcodesResponse> resp = controller.encodeCharcodes(request(" "));
EncodeCharcodesResponse body = resp.getBody();
assertThat(body).isNotNull();
assertThat(body.getError()).isNull();
// The space must be reported missing, NOT handed back as a charcode
// (0x20) the frontend would reuse into the „ glyph.
assertThat(body.getMissing()).containsExactly(" ");
assertThat(body.getCharcodes()).isNullOrEmpty();
}
@Test
void realCharsEncodeWhileWhitespaceStaysAGap() throws Exception {
PdfTextEditorCharcodeController controller = controller();
// "M M" - both M's must encode to real charcodes; only the space is a gap.
ResponseEntity<EncodeCharcodesResponse> resp = controller.encodeCharcodes(request("M M"));
EncodeCharcodesResponse body = resp.getBody();
assertThat(body).isNotNull();
assertThat(body.getError()).isNull();
assertThat(body.getCharcodes()).as("both M glyphs encode").hasSize(2);
assertThat(body.getMissing()).containsExactly(" ");
}
@Test
void tabAndNewlineAreAlsoTreatedAsGaps() throws Exception {
PdfTextEditorCharcodeController controller = controller();
ResponseEntity<EncodeCharcodesResponse> resp = controller.encodeCharcodes(request("\t\n"));
EncodeCharcodesResponse body = resp.getBody();
assertThat(body).isNotNull();
assertThat(body.getMissing()).containsExactly("\t", "\n");
assertThat(body.getCharcodes()).isNullOrEmpty();
}
/**
* A page with two fonts that BOTH render 'A'. {@code fontName} must select which one to encode
* against - the cross-font fix. Without it the first font in resources order won wins and a
* cross-font edit got the wrong font's charcode.
*/
private static String twoFontBase64() throws Exception {
try (PDDocument doc = new PDDocument()) {
PDPage page = new PDPage();
doc.addPage(page);
PDType1Font helvetica = new PDType1Font(Standard14Fonts.FontName.HELVETICA);
PDType1Font times = new PDType1Font(Standard14Fonts.FontName.TIMES_ROMAN);
try (PDPageContentStream cs = new PDPageContentStream(doc, page)) {
cs.beginText();
cs.setFont(helvetica, 12);
cs.newLineAtOffset(72, 720);
cs.showText("A");
cs.endText();
cs.beginText();
cs.setFont(times, 12);
cs.newLineAtOffset(72, 700);
cs.showText("A");
cs.endText();
}
ByteArrayOutputStream bos = new ByteArrayOutputStream();
doc.save(bos);
return Base64.getEncoder().encodeToString(bos.toByteArray());
}
}
private static EncodeCharcodesRequest twoFontRequest(String fontName) throws Exception {
EncodeCharcodesRequest req = new EncodeCharcodesRequest();
req.setPdfBase64(twoFontBase64());
req.setPageIndex(0);
req.setLocatorChar("A");
req.setFontName(fontName);
req.setText("A");
return req;
}
@Test
void fontNameDisambiguatesBetweenTwoFontsRenderingTheSameChar() throws Exception {
PdfTextEditorCharcodeController controller = controller();
// Targeting Times-Roman must encode against Times-Roman, not whichever
// font happens to appear first in the page's font resources.
EncodeCharcodesResponse times =
controller.encodeCharcodes(twoFontRequest("Times-Roman")).getBody();
assertThat(times).isNotNull();
assertThat(times.getError()).isNull();
assertThat(times.getNote()).contains("Times-Roman");
assertThat(times.getCharcodes()).hasSize(1);
// Targeting Helvetica must encode against Helvetica.
EncodeCharcodesResponse helv =
controller.encodeCharcodes(twoFontRequest("Helvetica")).getBody();
assertThat(helv).isNotNull();
assertThat(helv.getError()).isNull();
assertThat(helv.getNote()).contains("Helvetica");
assertThat(helv.getCharcodes()).hasSize(1);
}
@Test
void unknownFontNameReportsNoFontInsteadOfWrongFont() throws Exception {
PdfTextEditorCharcodeController controller = controller();
// A name that matches no font on the page must NOT silently encode
// against a different font: the frontend writes the returned charcodes
// into the NAMED font's text object, so a first-match fallback would
// bake wrong glyphs. It must report failure so the caller falls back.
EncodeCharcodesResponse body =
controller.encodeCharcodes(twoFontRequest("DoesNotExist")).getBody();
assertThat(body).isNotNull();
assertThat(body.getError()).contains("no font");
assertThat(body.getCharcodes()).isNull();
}
@Test
void missingRequiredFieldsReturns400() {
EncodeCharcodesRequest req = new EncodeCharcodesRequest();
req.setPdfBase64("AAAA");
req.setLocatorChar("M");
// text is null
ResponseEntity<EncodeCharcodesResponse> resp = controller().encodeCharcodes(req);
assertThat(resp.getStatusCode().value()).isEqualTo(400);
assertThat(resp.getBody()).isNotNull();
assertThat(resp.getBody().getError()).isEqualTo("missing required fields");
}
@Test
void invalidBase64Returns400() {
EncodeCharcodesRequest req = new EncodeCharcodesRequest();
req.setPdfBase64("!!!notbase64!!!");
req.setLocatorChar("M");
req.setText("M");
ResponseEntity<EncodeCharcodesResponse> resp = controller().encodeCharcodes(req);
assertThat(resp.getStatusCode().value()).isEqualTo(400);
assertThat(resp.getBody()).isNotNull();
assertThat(resp.getBody().getError()).isEqualTo("pdfBase64 is not valid base64");
}
@Test
void pageIndexOutOfRangeReturns400() throws Exception {
EncodeCharcodesRequest req = request("M");
req.setPageIndex(999);
ResponseEntity<EncodeCharcodesResponse> resp = controller().encodeCharcodes(req);
assertThat(resp.getStatusCode().value()).isEqualTo(400);
assertThat(resp.getBody()).isNotNull();
assertThat(resp.getBody().getError()).isEqualTo("pageIndex out of range");
}
@Test
void nonPdfBytesReturnsGenericError() {
EncodeCharcodesRequest req = new EncodeCharcodesRequest();
req.setPdfBase64(Base64.getEncoder().encodeToString("not a pdf".getBytes()));
req.setLocatorChar("M");
req.setText("M");
// Must not throw, and must not leak the raw PDFBox parser message.
ResponseEntity<EncodeCharcodesResponse> resp = controller().encodeCharcodes(req);
assertThat(resp.getStatusCode().is4xxClientError()).isTrue();
assertThat(resp.getBody()).isNotNull();
assertThat(resp.getBody().getError()).isEqualTo("failed to load PDF");
}
@Test
void absentLocatorCharReturns200WithError() throws Exception {
// U+FFFF never appears in the document, so no font matches.
ResponseEntity<EncodeCharcodesResponse> resp =
controller().encodeCharcodes(requestWithLocator("￿"));
assertThat(resp.getStatusCode().value()).isEqualTo(200);
EncodeCharcodesResponse body = resp.getBody();
assertThat(body).isNotNull();
assertThat(body.getError()).isNotNull();
assertThat(body.getCharcodes()).isNull();
}
@Test
void oversizePdfRejected() {
EncodeCharcodesRequest req = new EncodeCharcodesRequest();
// A base64 string long enough that length/4*3 exceeds the 100MB cap, without
// ever allocating the decoded bytes (the guard runs before decode).
char[] huge = new char[140 * 1024 * 1024];
java.util.Arrays.fill(huge, 'A');
req.setPdfBase64(new String(huge));
req.setLocatorChar("M");
req.setText("M");
ResponseEntity<EncodeCharcodesResponse> resp = controller().encodeCharcodes(req);
assertThat(resp.getStatusCode().value()).isEqualTo(413);
assertThat(resp.getBody()).isNotNull();
assertThat(resp.getBody().getError()).isEqualTo("pdf too large");
}
private static EncodeCharcodesRequest requestWithLocator(String locator) throws Exception {
EncodeCharcodesRequest req = request("M");
req.setLocatorChar(locator);
return req;
}
/**
* Build a page whose resources declare {@code filler} fonts that do NOT render 'A' (Symbol /
* ZapfDingbats have non-Latin encodings) plus, optionally, a trailing Helvetica that does. The
* Standard14 probe upper bound is 256 so each scan is cheap.
*/
private static String manyFontsBase64(int filler, boolean trailingTarget) throws Exception {
try (PDDocument doc = new PDDocument()) {
PDPage page = new PDPage();
doc.addPage(page);
org.apache.pdfbox.pdmodel.PDResources resources =
new org.apache.pdfbox.pdmodel.PDResources();
for (int n = 0; n < filler; n++) {
Standard14Fonts.FontName fn =
(n % 2 == 0)
? Standard14Fonts.FontName.SYMBOL
: Standard14Fonts.FontName.ZAPF_DINGBATS;
resources.put(
org.apache.pdfbox.cos.COSName.getPDFName("Ff" + n), new PDType1Font(fn));
}
if (trailingTarget) {
resources.put(
org.apache.pdfbox.cos.COSName.getPDFName("Target"),
new PDType1Font(Standard14Fonts.FontName.HELVETICA));
}
page.setResources(resources);
ByteArrayOutputStream bos = new ByteArrayOutputStream();
doc.save(bos);
return Base64.getEncoder().encodeToString(bos.toByteArray());
}
}
private static EncodeCharcodesRequest manyFontsRequest(String base64) {
EncodeCharcodesRequest req = new EncodeCharcodesRequest();
req.setPdfBase64(base64);
req.setPageIndex(0);
req.setLocatorChar("A");
req.setText("A");
return req;
}
@Test
void targetFontFoundAmongManyFonts() throws Exception {
// 60 non-matching fonts then the Helvetica target, all within the 64-font cap.
ResponseEntity<EncodeCharcodesResponse> resp =
controller().encodeCharcodes(manyFontsRequest(manyFontsBase64(60, true)));
assertThat(resp.getStatusCode().value()).isEqualTo(200);
EncodeCharcodesResponse body = resp.getBody();
assertThat(body).isNotNull();
assertThat(body.getError()).isNull();
assertThat(body.getCharcodes()).hasSize(1);
}
@Test
void targetBeyondFontCapReturnsGracefulNoFont() throws Exception {
// 64 non-matching fonts then the target at position 65 - the scan cap stops
// before reaching it, so we get a graceful no-font error rather than a full scan.
ResponseEntity<EncodeCharcodesResponse> resp =
controller().encodeCharcodes(manyFontsRequest(manyFontsBase64(64, true)));
assertThat(resp.getStatusCode().value()).isEqualTo(200);
EncodeCharcodesResponse body = resp.getBody();
assertThat(body).isNotNull();
assertThat(body.getError()).isNotNull();
assertThat(body.getCharcodes()).isNull();
}
// Same-family sibling subsets. One document can embed several subsets of
// one family, each re-encoded by order of first glyph use, so a letter has
// a different charcode in each ("R" = 0x21 in one, 0x22 in its sibling).
// FPDFFont_GetBaseFontName strips the "ABCDEF+" tag, so a name-based
// lookup cannot tell them apart and borrows the wrong subset's codes.
//
// The doc below mirrors that with two TrueType subsets differing only by
// subset tag. PUA code points keep it deterministic: font.encode() cannot
// resolve them by glyph name, so the charcode can only come from the
// selected font's ToUnicode reverse map - proving WHICH font was picked.
private static final String PUA = "";
/** ToUnicode CMap mapping each supplied charcode to a BMP code point. */
private static byte[] toUnicodeCmap(int[][] codeToUnicode) {
StringBuilder sb =
new StringBuilder(
"""
/CIDInit /ProcSet findresource begin
12 dict begin
begincmap
/CIDSystemInfo << /Registry (Adobe) /Ordering (UCS) /Supplement 0 >> def
/CMapName /Adobe-Identity-UCS def
/CMapType 2 def
1 begincodespacerange
<00><FF>
endcodespacerange
""");
sb.append(codeToUnicode.length).append(" beginbfchar\n");
for (int[] pair : codeToUnicode) {
sb.append(String.format("<%02X><%04X>%n", pair[0], pair[1]));
}
sb.append(
"""
endbfchar
endcmap
CMapName currentdict /CMap defineresource pop
end
end
""");
return sb.toString().getBytes(java.nio.charset.StandardCharsets.US_ASCII);
}
private static org.apache.pdfbox.cos.COSDictionary subsetFontDict(
PDDocument doc, String baseName, byte[] fontProgram, byte[] toUnicode)
throws Exception {
org.apache.pdfbox.cos.COSDictionary font = new org.apache.pdfbox.cos.COSDictionary();
font.setItem(org.apache.pdfbox.cos.COSName.TYPE, org.apache.pdfbox.cos.COSName.FONT);
font.setItem(
org.apache.pdfbox.cos.COSName.SUBTYPE, org.apache.pdfbox.cos.COSName.TRUE_TYPE);
if (baseName != null) {
font.setName(org.apache.pdfbox.cos.COSName.BASE_FONT, baseName);
}
font.setInt(org.apache.pdfbox.cos.COSName.FIRST_CHAR, 0x21);
font.setInt(org.apache.pdfbox.cos.COSName.LAST_CHAR, 0x22);
org.apache.pdfbox.cos.COSArray widths = new org.apache.pdfbox.cos.COSArray();
widths.add(org.apache.pdfbox.cos.COSInteger.get(500));
widths.add(org.apache.pdfbox.cos.COSInteger.get(500));
font.setItem(org.apache.pdfbox.cos.COSName.WIDTHS, widths);
org.apache.pdfbox.cos.COSDictionary fd = new org.apache.pdfbox.cos.COSDictionary();
fd.setItem(org.apache.pdfbox.cos.COSName.TYPE, org.apache.pdfbox.cos.COSName.FONT_DESC);
if (baseName != null) {
fd.setName(org.apache.pdfbox.cos.COSName.FONT_NAME, baseName);
}
fd.setInt(org.apache.pdfbox.cos.COSName.FLAGS, 4);
fd.setItem(
org.apache.pdfbox.cos.COSName.FONT_BBOX,
new org.apache.pdfbox.pdmodel.common.PDRectangle(0, 0, 1000, 1000).getCOSArray());
fd.setInt(org.apache.pdfbox.cos.COSName.ITALIC_ANGLE, 0);
fd.setInt(org.apache.pdfbox.cos.COSName.ASCENT, 800);
fd.setInt(org.apache.pdfbox.cos.COSName.DESCENT, -200);
fd.setInt(org.apache.pdfbox.cos.COSName.CAP_HEIGHT, 700);
fd.setInt(org.apache.pdfbox.cos.COSName.STEM_V, 80);
if (fontProgram != null) {
org.apache.pdfbox.pdmodel.common.PDStream ff2 =
new org.apache.pdfbox.pdmodel.common.PDStream(
doc, new java.io.ByteArrayInputStream(fontProgram));
ff2.getCOSObject().setInt(org.apache.pdfbox.cos.COSName.LENGTH1, fontProgram.length);
fd.setItem(org.apache.pdfbox.cos.COSName.FONT_FILE2, ff2.getCOSObject());
}
font.setItem(org.apache.pdfbox.cos.COSName.FONT_DESC, fd);
org.apache.pdfbox.pdmodel.common.PDStream tu =
new org.apache.pdfbox.pdmodel.common.PDStream(
doc, new java.io.ByteArrayInputStream(toUnicode));
font.setItem(org.apache.pdfbox.cos.COSName.getPDFName("ToUnicode"), tu.getCOSObject());
return font;
}
// Distinct fake font programs - hashing distinguishes the subsets by these bytes.
private static final byte[] PROGRAM_A =
"fake-ttf-program-A".getBytes(java.nio.charset.StandardCharsets.US_ASCII);
private static final byte[] PROGRAM_B =
"fake-ttf-program-B".getBytes(java.nio.charset.StandardCharsets.US_ASCII);
/**
* Two sibling subsets of "FakeGaramond" whose ToUnicode maps give U+E000 DIFFERENT charcodes:
* 0x22 in subset A (AAAAAC+), 0x21 in subset B (AAAAAG+) - exactly the CV's shifted-code
* layout. {@code includeSecond=false} keeps only subset A for the unambiguous-fallback case.
*/
private static String siblingSubsetsBase64(boolean includeSecond) throws Exception {
try (PDDocument doc = new PDDocument()) {
PDPage page = new PDPage();
doc.addPage(page);
org.apache.pdfbox.cos.COSDictionary fonts = new org.apache.pdfbox.cos.COSDictionary();
fonts.setItem(
org.apache.pdfbox.cos.COSName.getPDFName("TTA"),
subsetFontDict(
doc,
"AAAAAC+FakeGaramond",
PROGRAM_A,
toUnicodeCmap(new int[][] {{0x21, 0xE001}, {0x22, 0xE000}})));
if (includeSecond) {
fonts.setItem(
org.apache.pdfbox.cos.COSName.getPDFName("TTB"),
subsetFontDict(
doc,
"AAAAAG+FakeGaramond",
PROGRAM_B,
toUnicodeCmap(new int[][] {{0x21, 0xE000}, {0x22, 0xE002}})));
}
org.apache.pdfbox.pdmodel.PDResources resources =
new org.apache.pdfbox.pdmodel.PDResources();
resources.getCOSObject().setItem(org.apache.pdfbox.cos.COSName.FONT, fonts);
page.setResources(resources);
ByteArrayOutputStream bos = new ByteArrayOutputStream();
doc.save(bos);
return Base64.getEncoder().encodeToString(bos.toByteArray());
}
}
private static String sha256Hex(byte[] bytes) throws Exception {
byte[] digest = java.security.MessageDigest.getInstance("SHA-256").digest(bytes);
StringBuilder sb = new StringBuilder();
for (byte b : digest) sb.append(String.format("%02x", b));
return sb.toString();
}
private static EncodeCharcodesRequest siblingRequest(
String base64, String fontName, String fontSha256) {
EncodeCharcodesRequest req = new EncodeCharcodesRequest();
req.setPdfBase64(base64);
req.setPageIndex(0);
req.setLocatorChar(PUA);
req.setFontName(fontName);
req.setFontSha256(fontSha256);
req.setText(PUA);
return req;
}
@Test
void fontProgramHashSelectsTheExactSubset() throws Exception {
String base64 = siblingSubsetsBase64(true);
PdfTextEditorCharcodeController controller = controller();
// Both requests carry the SAME tag-stripped name PDFium reports ("FakeGaramond"),
// so only the program hash can tell the subsets apart.
EncodeCharcodesResponse viaA =
controller
.encodeCharcodes(
siblingRequest(base64, "FakeGaramond", sha256Hex(PROGRAM_A)))
.getBody();
assertThat(viaA).isNotNull();
assertThat(viaA.getError()).isNull();
assertThat(viaA.getNote()).contains("AAAAAC+FakeGaramond");
assertThat(viaA.getCharcodes()).containsExactly(0x22L);
EncodeCharcodesResponse viaB =
controller
.encodeCharcodes(
siblingRequest(base64, "FakeGaramond", sha256Hex(PROGRAM_B)))
.getBody();
assertThat(viaB).isNotNull();
assertThat(viaB.getError()).isNull();
assertThat(viaB.getNote()).contains("AAAAAG+FakeGaramond");
assertThat(viaB.getCharcodes()).containsExactly(0x21L);
}
@Test
void ambiguousStrippedNameRefusesToGuessBetweenSiblingSubsets() throws Exception {
// No hash, and the tag-stripped name matches BOTH subsets which both render the
// locator char. Guessing here is what scrambled "RUSSELL W. MANGUM III" into
// "US EEL W. MANGS M III" - the sibling's codes hit different glyphs. The
// backend must refuse so the frontend takes its safe fallback.
EncodeCharcodesResponse body =
controller()
.encodeCharcodes(
siblingRequest(siblingSubsetsBase64(true), "FakeGaramond", null))
.getBody();
assertThat(body).isNotNull();
assertThat(body.getError()).contains("no font");
assertThat(body.getCharcodes()).isNull();
}
@Test
void exactTaggedNameStillSelectsItsSubset() throws Exception {
// A caller that DOES know the full tagged /BaseFont name keeps working.
EncodeCharcodesResponse body =
controller()
.encodeCharcodes(
siblingRequest(
siblingSubsetsBase64(true), "AAAAAG+FakeGaramond", null))
.getBody();
assertThat(body).isNotNull();
assertThat(body.getError()).isNull();
assertThat(body.getNote()).contains("AAAAAG+FakeGaramond");
assertThat(body.getCharcodes()).containsExactly(0x21L);
}
@Test
void strippedNameStillWorksWhenUnambiguous() throws Exception {
// With a SINGLE subset on the page, the tag-stripped name (what PDFium
// reports) must keep resolving - the ambiguity guard only bites when
// two+ siblings could answer.
EncodeCharcodesResponse body =
controller()
.encodeCharcodes(
siblingRequest(siblingSubsetsBase64(false), "FakeGaramond", null))
.getBody();
assertThat(body).isNotNull();
assertThat(body.getError()).isNull();
assertThat(body.getNote()).contains("AAAAAC+FakeGaramond");
assertThat(body.getCharcodes()).containsExactly(0x22L);
}
@Test
void staleHashFallsBackToNameMatching() throws Exception {
// A hash matching NO font on the page (e.g. PDFium handed back a substitute
// font's bytes) must not brick the request: name matching still runs, and an
// exact tagged name resolves.
EncodeCharcodesResponse body =
controller()
.encodeCharcodes(
siblingRequest(
siblingSubsetsBase64(true),
"AAAAAC+FakeGaramond",
"0000000000000000000000000000000000000000000000000000000000000000"))
.getBody();
assertThat(body).isNotNull();
assertThat(body.getError()).isNull();
assertThat(body.getNote()).contains("AAAAAC+FakeGaramond");
assertThat(body.getCharcodes()).containsExactly(0x22L);
}
private static final String PUA_E000 = "";
private static final String PUA_E002 = "";
private static final byte[] SHARED_PROGRAM =
"fake-ttf-program-shared".getBytes(java.nio.charset.StandardCharsets.US_ASCII);
private static String cacheIdentityPairBase64(String baseName, byte[] program)
throws Exception {
try (PDDocument doc = new PDDocument()) {
PDPage page = new PDPage();
doc.addPage(page);
org.apache.pdfbox.cos.COSDictionary fonts = new org.apache.pdfbox.cos.COSDictionary();
fonts.setItem(
org.apache.pdfbox.cos.COSName.getPDFName("C1"),
subsetFontDict(
doc,
baseName,
program,
toUnicodeCmap(new int[][] {{0x21, 0xE001}, {0x22, 0xE000}})));
fonts.setItem(
org.apache.pdfbox.cos.COSName.getPDFName("C2"),
subsetFontDict(
doc,
baseName,
program,
toUnicodeCmap(new int[][] {{0x21, 0xE002}, {0x22, 0xE003}})));
org.apache.pdfbox.pdmodel.PDResources resources =
new org.apache.pdfbox.pdmodel.PDResources();
resources.getCOSObject().setItem(org.apache.pdfbox.cos.COSName.FONT, fonts);
page.setResources(resources);
ByteArrayOutputStream bos = new ByteArrayOutputStream();
doc.save(bos);
return Base64.getEncoder().encodeToString(bos.toByteArray());
}
}
private static EncodeCharcodesRequest cacheIdentityRequest(
String base64, String locator, String fontName, String fontSha256) {
EncodeCharcodesRequest req = new EncodeCharcodesRequest();
req.setPdfBase64(base64);
req.setPageIndex(0);
req.setLocatorChar(locator);
req.setFontName(fontName);
req.setFontSha256(fontSha256);
req.setText(locator);
return req;
}
@Test
void unnamedFontsSharingOneProgramDoNotShareACachedMap() throws Exception {
String base64 = cacheIdentityPairBase64(null, SHARED_PROGRAM);
String sha = sha256Hex(SHARED_PROGRAM);
PdfTextEditorCharcodeController controller = controller();
EncodeCharcodesResponse first =
controller
.encodeCharcodes(cacheIdentityRequest(base64, PUA_E000, null, sha))
.getBody();
assertThat(first).isNotNull();
assertThat(first.getError()).isNull();
assertThat(first.getCharcodes()).containsExactly(0x22L);
EncodeCharcodesResponse second =
controller
.encodeCharcodes(cacheIdentityRequest(base64, PUA_E002, null, sha))
.getBody();
assertThat(second).isNotNull();
assertThat(second.getError()).isNull();
assertThat(second.getMissing()).isNullOrEmpty();
assertThat(second.getCharcodes())
.as("second font must not be served the first font's cached map")
.containsExactly(0x21L);
}
@Test
void fontsSharingOneNameDoNotShareACachedMap() throws Exception {
String base64 = cacheIdentityPairBase64("SharedName", null);
PdfTextEditorCharcodeController controller = controller();
EncodeCharcodesResponse first =
controller
.encodeCharcodes(cacheIdentityRequest(base64, PUA_E000, "SharedName", null))
.getBody();
assertThat(first).isNotNull();
assertThat(first.getError()).isNull();
assertThat(first.getCharcodes()).containsExactly(0x22L);
EncodeCharcodesResponse second =
controller
.encodeCharcodes(cacheIdentityRequest(base64, PUA_E002, "SharedName", null))
.getBody();
assertThat(second).isNotNull();
assertThat(second.getError()).isNull();
assertThat(second.getMissing()).isNullOrEmpty();
assertThat(second.getCharcodes())
.as("same-name fonts must not share one cached map")
.containsExactly(0x21L);
}
private static String formXObjectFontBase64() throws Exception {
try (PDDocument doc = new PDDocument()) {
PDPage page = new PDPage();
doc.addPage(page);
org.apache.pdfbox.pdmodel.graphics.form.PDFormXObject outer =
new org.apache.pdfbox.pdmodel.graphics.form.PDFormXObject(doc);
outer.setBBox(new org.apache.pdfbox.pdmodel.common.PDRectangle(0, 0, 200, 200));
org.apache.pdfbox.pdmodel.graphics.form.PDFormXObject inner =
new org.apache.pdfbox.pdmodel.graphics.form.PDFormXObject(doc);
inner.setBBox(new org.apache.pdfbox.pdmodel.common.PDRectangle(0, 0, 100, 100));
org.apache.pdfbox.pdmodel.PDResources innerResources =
new org.apache.pdfbox.pdmodel.PDResources();
innerResources.put(
org.apache.pdfbox.cos.COSName.getPDFName("F1"),
new PDType1Font(Standard14Fonts.FontName.HELVETICA));
inner.setResources(innerResources);
org.apache.pdfbox.pdmodel.PDResources outerResources =
new org.apache.pdfbox.pdmodel.PDResources();
outerResources.put(org.apache.pdfbox.cos.COSName.getPDFName("Fm1"), inner);
outer.setResources(outerResources);
org.apache.pdfbox.pdmodel.PDResources pageResources =
new org.apache.pdfbox.pdmodel.PDResources();
pageResources.put(org.apache.pdfbox.cos.COSName.getPDFName("Fm0"), outer);
page.setResources(pageResources);
ByteArrayOutputStream bos = new ByteArrayOutputStream();
doc.save(bos);
return Base64.getEncoder().encodeToString(bos.toByteArray());
}
}
private static String cyclicFormXObjectsBase64() throws Exception {
try (PDDocument doc = new PDDocument()) {
PDPage page = new PDPage();
doc.addPage(page);
org.apache.pdfbox.pdmodel.graphics.form.PDFormXObject formA =
new org.apache.pdfbox.pdmodel.graphics.form.PDFormXObject(doc);
formA.setBBox(new org.apache.pdfbox.pdmodel.common.PDRectangle(0, 0, 100, 100));
org.apache.pdfbox.pdmodel.graphics.form.PDFormXObject formB =
new org.apache.pdfbox.pdmodel.graphics.form.PDFormXObject(doc);
formB.setBBox(new org.apache.pdfbox.pdmodel.common.PDRectangle(0, 0, 100, 100));
org.apache.pdfbox.pdmodel.PDResources resA =
new org.apache.pdfbox.pdmodel.PDResources();
org.apache.pdfbox.pdmodel.PDResources resB =
new org.apache.pdfbox.pdmodel.PDResources();
resA.put(org.apache.pdfbox.cos.COSName.getPDFName("Self"), formA);
resA.put(org.apache.pdfbox.cos.COSName.getPDFName("Fb"), formB);
resB.put(org.apache.pdfbox.cos.COSName.getPDFName("Fa"), formA);
resB.put(
org.apache.pdfbox.cos.COSName.getPDFName("F1"),
new PDType1Font(Standard14Fonts.FontName.HELVETICA));
formA.setResources(resA);
formB.setResources(resB);
org.apache.pdfbox.pdmodel.PDResources pageResources =
new org.apache.pdfbox.pdmodel.PDResources();
pageResources.put(org.apache.pdfbox.cos.COSName.getPDFName("Fm0"), formA);
page.setResources(pageResources);
ByteArrayOutputStream bos = new ByteArrayOutputStream();
doc.save(bos);
return Base64.getEncoder().encodeToString(bos.toByteArray());
}
}
@Test
void fontReachableOnlyThroughAFormXObjectIsFound() throws Exception {
ResponseEntity<EncodeCharcodesResponse> resp =
controller().encodeCharcodes(manyFontsRequest(formXObjectFontBase64()));
assertThat(resp.getStatusCode().value()).isEqualTo(200);
EncodeCharcodesResponse body = resp.getBody();
assertThat(body).isNotNull();
assertThat(body.getError()).isNull();
assertThat(body.getNote()).contains("Helvetica");
assertThat(body.getCharcodes()).containsExactly((long) 'A');
}
@Test
@org.junit.jupiter.api.Timeout(60)
void cyclicFormXObjectResourcesTerminate() throws Exception {
ResponseEntity<EncodeCharcodesResponse> resp =
controller().encodeCharcodes(manyFontsRequest(cyclicFormXObjectsBase64()));
assertThat(resp.getStatusCode().value()).isEqualTo(200);
EncodeCharcodesResponse body = resp.getBody();
assertThat(body).isNotNull();
assertThat(body.getError()).isNull();
assertThat(body.getCharcodes()).containsExactly((long) 'A');
}
}
@@ -0,0 +1,340 @@
package stirling.software.SPDF.controller.api;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.HashSet;
import java.util.Set;
import java.util.TreeSet;
import org.apache.pdfbox.Loader;
import org.apache.pdfbox.contentstream.PDFStreamEngine;
import org.apache.pdfbox.contentstream.operator.state.Concatenate;
import org.apache.pdfbox.contentstream.operator.state.Restore;
import org.apache.pdfbox.contentstream.operator.state.Save;
import org.apache.pdfbox.contentstream.operator.state.SetGraphicsStateParameters;
import org.apache.pdfbox.contentstream.operator.state.SetMatrix;
import org.apache.pdfbox.contentstream.operator.text.BeginText;
import org.apache.pdfbox.contentstream.operator.text.EndText;
import org.apache.pdfbox.contentstream.operator.text.SetFontAndSize;
import org.apache.pdfbox.contentstream.operator.text.SetTextHorizontalScaling;
import org.apache.pdfbox.contentstream.operator.text.SetTextLeading;
import org.apache.pdfbox.contentstream.operator.text.SetTextRenderingMode;
import org.apache.pdfbox.contentstream.operator.text.SetTextRise;
import org.apache.pdfbox.contentstream.operator.text.SetWordSpacing;
import org.apache.pdfbox.contentstream.operator.text.ShowText;
import org.apache.pdfbox.cos.COSBase;
import org.apache.pdfbox.cos.COSDictionary;
import org.apache.pdfbox.cos.COSName;
import org.apache.pdfbox.cos.COSStream;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDPage;
import org.apache.pdfbox.pdmodel.PDResources;
import org.apache.pdfbox.pdmodel.font.PDFont;
import org.apache.pdfbox.pdmodel.font.PDFontDescriptor;
import org.apache.pdfbox.pdmodel.font.PDType3CharProc;
import org.apache.pdfbox.pdmodel.font.PDType3Font;
import org.junit.jupiter.api.Disabled;
import org.junit.jupiter.api.Test;
/**
* Diagnostic test: enumerate every font referenced by Sample.pdf and dump its subtype, encoding,
* ToUnicode, and embedded font program info. For Type3 fonts also dump CharProcs glyph names and
* the content stream of one glyph (the 'M' if present).
*
* <p>Not a real regression test - run with --tests SamplePdfFontDumpTest -i to see the stdout
* output.
*/
@Disabled(
"Diagnostic probe: dumps Sample.pdf font internals to stdout and asserts nothing. Kept for font debugging; run manually.")
public class SamplePdfFontDumpTest {
private static final Path SAMPLE =
Paths.get(System.getProperty("user.dir"))
.getParent()
.getParent()
.resolve("frontend/editor/public/samples/Sample.pdf");
@Test
public void dumpFonts() throws IOException {
byte[] pdfBytes = Files.readAllBytes(SAMPLE);
try (PDDocument doc = Loader.loadPDF(pdfBytes)) {
int numPages = doc.getNumberOfPages();
System.out.println("Sample.pdf has " + numPages + " pages.");
Set<COSDictionary> seenFontDicts = new HashSet<>();
for (int p = 0; p < numPages; p++) {
PDPage page = doc.getPage(p);
System.out.println("\n=== Page " + p + " ===");
PDResources resources = page.getResources();
if (resources == null) {
System.out.println(" (no resources)");
continue;
}
for (COSName fontName : resources.getFontNames()) {
PDFont font;
try {
font = resources.getFont(fontName);
} catch (IOException e) {
System.out.println(
" Font "
+ fontName.getName()
+ ": failed to load - "
+ e.getMessage());
continue;
}
if (font == null) continue;
COSDictionary dict = font.getCOSObject();
if (!seenFontDicts.add(dict)) {
System.out.println(
" Font " + fontName.getName() + " -> already seen above");
continue;
}
dumpFont(fontName.getName(), font);
}
}
// Scan: for every text-show operation, record per-font (charcode, unicode) pairs.
System.out.println("\n=== All (font, charcode, unicode) seen on page ===");
for (int p = 0; p < numPages; p++) {
PDPage page = doc.getPage(p);
AllCharsScanner scanner = new AllCharsScanner();
scanner.processPage(page);
System.out.println("\nPage " + p + ":");
for (var entry : scanner.perFont.entrySet()) {
PDFont font = entry.getKey();
var seen = entry.getValue();
System.out.println(" Font " + font.getName() + " " + font.getSubType() + ":");
var sortedSeen = new java.util.TreeMap<Integer, String>(seen);
for (var s : sortedSeen.entrySet()) {
System.out.println(
" charcode 0x"
+ Integer.toHexString(s.getKey())
+ " ("
+ s.getKey()
+ ") -> '"
+ s.getValue()
+ "'");
}
}
}
// Confirm font.encode() works for Type3 fonts.
System.out.println("\n=== Can we encode existing chars in F27/F28? ===");
PDPage page0 = doc.getPage(0);
PDResources r0 = page0.getResources();
for (String fname : new String[] {"F27", "F28"}) {
PDFont f = r0.getFont(COSName.getPDFName(fname));
if (f == null) {
System.out.println(" " + fname + ": NOT FOUND on page 0");
continue;
}
System.out.println(" " + fname + ": " + f.getClass().getSimpleName());
for (String ch : new String[] {"M", "0", "1", "+", "Z", "a"}) {
try {
byte[] enc = f.encode(ch);
StringBuilder sb = new StringBuilder();
for (byte b : enc) sb.append(String.format("%02X ", b & 0xff));
System.out.println(
" encode('" + ch + "') -> [" + sb.toString().trim() + "]");
} catch (Exception e) {
System.out.println(
" encode('"
+ ch
+ "') FAILED: "
+ e.getClass().getSimpleName()
+ " "
+ e.getMessage());
}
}
}
// Dump page 0 content stream so we can see how "10M+" is composed.
System.out.println("\n=== Page 0 RAW content stream (first 4kb) ===");
try (InputStream is = doc.getPage(0).getContents()) {
byte[] bytes = is.readAllBytes();
System.out.println("Total content stream size: " + bytes.length + " bytes");
String asStr = new String(bytes, StandardCharsets.ISO_8859_1);
int idx = asStr.indexOf("F27");
if (idx >= 0) {
int start = Math.max(0, idx - 100);
int end = Math.min(asStr.length(), idx + 2500);
System.out.println("--- F27 context ---");
System.out.println(asStr.substring(start, end));
System.out.println("---");
}
int idx2 = asStr.indexOf("F28");
if (idx2 >= 0) {
int start = Math.max(0, idx2 - 200);
int end = Math.min(asStr.length(), idx2 + 600);
System.out.println("--- F28 context ---");
System.out.println(asStr.substring(start, end));
System.out.println("---");
}
}
// Dump a CharProc for each font's first non-zero glyph, with focus on any 'M' or "0".
System.out.println("\n=== Sample CharProc dumps for Type3 fonts ===");
Set<COSDictionary> printed = new HashSet<>();
for (int p = 0; p < numPages; p++) {
PDPage page = doc.getPage(p);
PDResources resources = page.getResources();
if (resources == null) continue;
for (COSName fn : resources.getFontNames()) {
PDFont font = resources.getFont(fn);
if (!(font instanceof PDType3Font)) continue;
if (!printed.add(font.getCOSObject())) continue;
PDType3Font t3 = (PDType3Font) font;
// Iterate charcodes 0..255 looking for any that map to 'M' or '0' or '+'.
for (int cc = 0; cc < 256; cc++) {
String u = null;
try {
u = t3.toUnicode(cc);
} catch (Exception e) {
/* */
}
if (u == null) continue;
if (u.equals("M") || u.equals("0") || u.equals("+") || u.equals("1")) {
System.out.println(
"Page "
+ p
+ " font '"
+ fn.getName()
+ "' charcode "
+ cc
+ " maps to '"
+ u
+ "':");
dumpType3Glyph(t3, cc);
}
}
}
}
}
}
private void dumpFont(String resourceName, PDFont font) {
COSDictionary dict = font.getCOSObject();
String subtype = dict.getNameAsString(COSName.SUBTYPE);
String baseFont = dict.getNameAsString(COSName.BASE_FONT);
boolean hasEncoding = dict.containsKey(COSName.ENCODING);
boolean hasToUnicode = dict.containsKey(COSName.TO_UNICODE);
PDFontDescriptor descriptor = font.getFontDescriptor();
boolean hasEmbedded = false;
String embeddedKind = "none";
if (descriptor != null) {
COSDictionary dDict = descriptor.getCOSObject();
if (dDict.containsKey(COSName.FONT_FILE)) {
hasEmbedded = true;
embeddedKind = "FontFile (Type1)";
} else if (dDict.containsKey(COSName.FONT_FILE2)) {
hasEmbedded = true;
embeddedKind = "FontFile2 (TrueType)";
} else if (dDict.containsKey(COSName.FONT_FILE3)) {
hasEmbedded = true;
COSBase ff3 = dDict.getDictionaryObject(COSName.FONT_FILE3);
if (ff3 instanceof COSStream) {
String ff3Subtype = ((COSStream) ff3).getNameAsString(COSName.SUBTYPE);
embeddedKind = "FontFile3 (" + ff3Subtype + ")";
} else {
embeddedKind = "FontFile3";
}
}
}
System.out.println(
" Font resource '"
+ resourceName
+ "': base='"
+ baseFont
+ "' subtype="
+ subtype
+ " hasEncoding="
+ hasEncoding
+ " hasToUnicode="
+ hasToUnicode
+ " embedded="
+ hasEmbedded
+ " ("
+ embeddedKind
+ ")");
if (font instanceof PDType3Font) {
PDType3Font t3 = (PDType3Font) font;
COSDictionary charProcs = t3.getCharProcs();
int count = charProcs == null ? 0 : charProcs.size();
System.out.println(" Type3 CharProcs count = " + count);
if (charProcs != null) {
TreeSet<String> names = new TreeSet<>();
for (COSName k : charProcs.keySet()) names.add(k.getName());
System.out.println(" glyph names: " + names);
}
}
}
private void dumpType3Glyph(PDType3Font font, int charcode) throws IOException {
String name = font.getEncoding() != null ? font.getEncoding().getName(charcode) : null;
System.out.println(" Type3 charcode " + charcode + " -> glyph name '" + name + "'");
PDType3CharProc proc = font.getCharProc(charcode);
if (proc == null) {
System.out.println(" (no CharProc for that charcode)");
return;
}
COSStream stream = proc.getCOSObject();
byte[] raw;
try (InputStream is = stream.createInputStream()) {
raw = is.readAllBytes();
}
System.out.println(" CharProc content stream (" + raw.length + " bytes):");
System.out.println("---");
System.out.println(new String(raw, StandardCharsets.ISO_8859_1));
System.out.println("---");
}
/** Records every (font, charcode -> unicode) tuple seen on a page. */
static final class AllCharsScanner extends PDFStreamEngine {
final java.util.LinkedHashMap<PDFont, java.util.Map<Integer, String>> perFont =
new java.util.LinkedHashMap<>();
AllCharsScanner() {
addOperator(new BeginText(this));
addOperator(new EndText(this));
addOperator(new SetFontAndSize(this));
addOperator(new SetTextHorizontalScaling(this));
addOperator(new SetTextLeading(this));
addOperator(new SetTextRenderingMode(this));
addOperator(new SetTextRise(this));
addOperator(new SetWordSpacing(this));
addOperator(new SetMatrix(this));
addOperator(new Save(this));
addOperator(new Restore(this));
addOperator(new Concatenate(this));
addOperator(new SetGraphicsStateParameters(this));
addOperator(new ShowText(this));
}
@Override
protected void showText(byte[] string) throws IOException {
PDFont font = getGraphicsState().getTextState().getFont();
if (font == null) return;
var seen = perFont.computeIfAbsent(font, k -> new java.util.LinkedHashMap<>());
ByteArrayInputStream in = new ByteArrayInputStream(string);
while (in.available() > 0) {
int code;
try {
code = font.readCode(in);
} catch (IOException e) {
break;
}
String u;
try {
u = font.toUnicode(code);
} catch (RuntimeException e) {
u = null;
}
seen.putIfAbsent(code, u);
}
}
}
}
@@ -485,9 +485,9 @@ class PdfJsonFontServiceMoreTest {
class DetectExtra {
@Test
@DisplayName("detectFontFlavor recognises ttcf as cff and otf via OTTO")
@DisplayName("detectFontFlavor rejects ttcf collections and recognises otf via OTTO")
void detectFlavorExtra() {
assertEquals("cff", service.detectFontFlavor(new byte[] {0x74, 0x74, 0x63, 0x66}));
assertNull(service.detectFontFlavor(new byte[] {0x74, 0x74, 0x63, 0x66}));
List<byte[]> otfVariants = List.of(new byte[] {0x4F, 0x54, 0x54, 0x4F});
for (byte[] otf : otfVariants) {
assertEquals("otf", service.detectFontFlavor(otf));
@@ -57,10 +57,9 @@ class PdfJsonFontServiceTest {
}
@Test
void detectFontFlavor_cffSignature_returnsCff() {
// 0x74746366 = "ttcf"
byte[] cff = {0x74, 0x74, 0x63, 0x66};
assertEquals("cff", service.detectFontFlavor(cff));
void detectFontFlavor_ttcSignature_returnsNull() {
byte[] ttc = {0x74, 0x74, 0x63, 0x66};
assertNull(service.detectFontFlavor(ttc));
}
@Test
@@ -94,9 +93,9 @@ class PdfJsonFontServiceTest {
}
@Test
void detectTrueTypeFormat_cffSignature_returnsCff() {
byte[] cff = {0x74, 0x74, 0x63, 0x66};
assertEquals("cff", service.detectTrueTypeFormat(cff));
void detectTrueTypeFormat_ttcSignature_returnsNull() {
byte[] ttc = {0x74, 0x74, 0x63, 0x66};
assertNull(service.detectTrueTypeFormat(ttc));
}
@Test
@@ -18,6 +18,19 @@ public enum FailureActionId {
DISMISS(Execution.SERVER, "Dismiss"),
/**
* Open the failed operation in the client with its document, for the owner to run again
* themselves. Not a re-run: the settings are theirs to check first.
*/
OPEN_IN_TOOL(Execution.CLIENT, "Retry"),
/**
* Ask the owner for the password and unlock the document in their client. Re-running is implied
* rather than named: an id says what a caller must supply, and a {@link
* FailureActionSlot#RESOLUTION} runs the failed work again once it has it.
*/
DECRYPT(Execution.CLIENT, "Decrypt and retry"),
/** Open the document behind the incident, in whichever client can resolve its id. */
VIEW_FILE(Execution.CLIENT, "View file"),
@@ -0,0 +1,14 @@
package stirling.software.proprietary.failure;
/** Placement intent, not layout: the client promotes, knowing what it can actually run. */
public enum FailureActionSlot {
/** The action that resolves the failure. At most one per kind. */
RESOLUTION,
/** Offered alongside the resolution, for a caller the resolution is not aimed at. */
SECONDARY,
/** Available but folded away: correct, rarely what anyone wants to press next. */
OVERFLOW
}
@@ -1,8 +1,12 @@
package stirling.software.proprietary.failure;
import static stirling.software.proprietary.failure.FailureActionId.DECRYPT;
import static stirling.software.proprietary.failure.FailureActionId.DISMISS;
import static stirling.software.proprietary.failure.FailureActionId.OPEN_IN_TOOL;
import static stirling.software.proprietary.failure.FailureActionId.VIEW_FILE;
import static stirling.software.proprietary.failure.FailureActionId.VIEW_IN_PROCESSOR;
import static stirling.software.proprietary.failure.FailureActionSlot.OVERFLOW;
import static stirling.software.proprietary.failure.FailureActionSlot.SECONDARY;
import static stirling.software.proprietary.failure.FailureAudience.ANYONE_WHO_SEES;
import static stirling.software.proprietary.failure.FailureAudience.OWNER;
import static stirling.software.proprietary.failure.FailureAudience.TEAM_REVIEWER;
@@ -21,11 +25,8 @@ import lombok.AccessLevel;
import lombok.Getter;
/**
* The registry of failure kinds, described as data: a stable id, i18n keys and an English fallback
* like {@code ExceptionUtils.ErrorCode}, plus the facets a review surface needs.
*
* <p>A new kind ships as a registry entry plus copy. Each offer also says who it is for, since one
* incident is read both by whoever hit it and by whoever reviews after them.
* The registry of failure kinds as data: id, i18n keys, English fallback, plus the facets a review
* surface needs. A new kind ships as an entry plus copy; each offer says who it is for and where.
*/
@Getter
public enum FailureKind {
@@ -36,9 +37,12 @@ public enum FailureKind {
FailureScope.FILE,
errorCodes("E004"),
fallback("This document is password-protected, so the pipeline could not read it."),
offer(VIEW_FILE, OWNER),
offer(VIEW_IN_PROCESSOR, TEAM_REVIEWER),
offer(DISMISS, ANYONE_WHO_SEES)),
// The password is the fix; the owner's own document is the runner-up.
resolution(DECRYPT, OWNER),
global(VIEW_FILE, OWNER, SECONDARY),
global(VIEW_IN_PROCESSOR, TEAM_REVIEWER, OVERFLOW),
global(OPEN_IN_TOOL, OWNER, OVERFLOW),
global(DISMISS, ANYONE_WHO_SEES, OVERFLOW)),
UNKNOWN(
FailureStage.INTERNAL,
@@ -47,11 +51,11 @@ public enum FailureKind {
FailureScope.RUN,
noErrorCodes(),
fallback("This run failed for a reason Stirling does not yet recognise."),
// Same order as every other kind: declaration order is display order, so the document
// leads wherever it is offered rather than moving between failures.
offer(VIEW_FILE, OWNER),
offer(VIEW_IN_PROCESSOR, TEAM_REVIEWER),
offer(DISMISS, ANYONE_WHO_SEES));
// No known fix to declare, so a plain retry leads: these are often one-offs.
global(OPEN_IN_TOOL, OWNER, SECONDARY),
global(VIEW_FILE, OWNER, SECONDARY),
global(VIEW_IN_PROCESSOR, TEAM_REVIEWER, OVERFLOW),
global(DISMISS, ANYONE_WHO_SEES, OVERFLOW));
private static final String KEY_PREFIX = "portal.failures.kind.";
private static final String ACTION_KEY_PREFIX = "portal.failures.action.";
@@ -98,27 +102,37 @@ public enum FailureKind {
this.offers = List.of(offers);
}
/**
* One ordered list rather than ids plus parallel maps of audiences and labels, which could
* disagree with each other.
*
* @param labelKeySuffix key under {@code portal.failures.action.}, or null for the generic
* label
*/
private record Offer(FailureActionId id, FailureAudience audience, String labelKeySuffix) {}
/** One ordered list, not parallel maps of audiences, slots and labels that could disagree. */
private record Offer(
FailureActionId id,
FailureAudience audience,
FailureActionSlot slot,
String labelKeySuffix) {}
/** Declaration order is display order. */
private static Offer offer(FailureActionId id, FailureAudience audience) {
return new Offer(id, audience, null);
/** The action that fixes this kind. One per kind: needing two would make it two kinds. */
private static Offer resolution(FailureActionId id, FailureAudience audience) {
return new Offer(id, audience, FailureActionSlot.RESOLUTION, null);
}
/**
* As {@link #offer(FailureActionId, FailureAudience)}, but labelled by this kind's own wording
* where the shared one reads badly.
*/
private static Offer offer(
/** As {@link #resolution(FailureActionId, FailureAudience)}, with this kind's own wording. */
private static Offer resolution(
FailureActionId id, FailureAudience audience, String labelKeySuffix) {
return new Offer(id, audience, labelKeySuffix);
return new Offer(id, audience, FailureActionSlot.RESOLUTION, labelKeySuffix);
}
/** Not this kind's fix: an offer any kind can make, with the shared wording. */
private static Offer global(
FailureActionId id, FailureAudience audience, FailureActionSlot slot) {
return new Offer(id, audience, slot, null);
}
/** As above, with this kind's own wording where the shared one reads badly. */
private static Offer global(
FailureActionId id,
FailureAudience audience,
FailureActionSlot slot,
String labelKeySuffix) {
return new Offer(id, audience, slot, labelKeySuffix);
}
/**
@@ -157,21 +171,25 @@ public enum FailureKind {
return offers.stream().map(Offer::id).toList();
}
/**
* What this kind offers, in declaration order, each with its label resolved. What a review
* surface reads, so it never has to ask two separate questions about one offer.
*/
/** What this kind offers, in declaration order, each with label and placement resolved. */
public List<OfferedAction> getOfferedActions() {
return offers.stream()
.map(
offer ->
new OfferedAction(
offer.id(), labelKeyFor(offer.id()), offer.audience()))
offer.id(),
labelKeyFor(offer.id()),
offer.audience(),
offer.slot()))
.toList();
}
/** One action as a kind declares it: what to call it and who it is for. */
public record OfferedAction(FailureActionId id, String labelKey, FailureAudience audience) {}
/** One action as a kind declares it: what to call it, who it is for, where it wants to sit. */
public record OfferedAction(
FailureActionId id,
String labelKey,
FailureAudience audience,
FailureActionSlot slot) {}
/** Whether this kind offers {@code action}. The dispatch guard: see {@code FailureActionId}. */
public boolean declares(FailureActionId action) {
@@ -64,16 +64,17 @@ public interface FileRunEventRepository extends JpaRepository<FileRunEventEntity
int fold(@Param("id") String id, @Param("now") Instant now, @Param("detail") String detail);
/**
* Reopen a resolved incident whose failure has recurred. Guarded on the current status so only
* {@code RESOLVED} flips; a concurrent dismiss is never overwritten back to {@code NEW}.
* A recurrence reopens {@code RESOLVED} (the fix did not hold) and {@code FILE_REMOVED} (the
* document is back). Guarded, so a reviewer's {@code DISMISSED} is never overwritten.
*/
@Modifying(clearAutomatically = true)
@Transactional
@Query(
"update FileRunEventEntity e set"
+ " e.status = stirling.software.proprietary.failure.FileRunEventStatus.NEW,"
+ " e.statusActor = null, e.statusAt = null where e.id = :id and e.status ="
+ " stirling.software.proprietary.failure.FileRunEventStatus.RESOLVED")
+ " e.statusActor = null, e.statusAt = null where e.id = :id and e.status in"
+ " (stirling.software.proprietary.failure.FileRunEventStatus.RESOLVED,"
+ " stirling.software.proprietary.failure.FileRunEventStatus.FILE_REMOVED)")
int reopenIfResolved(@Param("id") String id);
/**
@@ -1,5 +1,9 @@
package stirling.software.proprietary.failure;
import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.HexFormat;
import java.util.List;
import java.util.Map;
@@ -171,6 +175,18 @@ public class FileRunEventService {
return action.execute(event, inputs == null ? Map.of() : inputs, currentActor());
}
/** Mark an incident resolved after a client's own retry worked. Idempotent. */
public FileRunEvent resolve(String eventId) {
FileRunEvent event = requireVisible(eventId);
// No terminal pre-check: the store's guarded UPDATE decides, rather than racing a read.
return store.applyStatusOnce(
event.id(),
event.teamId(),
FileRunEventStatus.RESOLVED,
currentActor(),
FileRunEventStatus.open());
}
/** "No such event" rather than a refusal, so trying does not confirm a colleague's exists. */
private FileRunEvent requireVisible(String eventId) {
ReadScope scope = readScope();
@@ -222,7 +238,8 @@ public class FileRunEventService {
boolean unattended,
boolean documentless) {
String reason = disabledReasonFor(offer.audience(), closed, unattended, documentless);
return new AvailableAction(offer.id(), offer.labelKey(), reason == null, reason);
return new AvailableAction(
offer.id(), offer.labelKey(), offer.slot(), reason == null, reason);
}
/** Closed wins over everything, then the owner-only reasons, most specific first. */
@@ -251,11 +268,38 @@ public class FileRunEventService {
};
}
/** Login disabled has no roles, so its one operator triages everything. */
private boolean reviewsTeam() {
/** Whether the caller triages the team's incidents, not only their own. Login disabled: all. */
public boolean reviewsTeam() {
return !enforced() || policyManagementAuthority.canEditPolicies();
}
/**
* An opaque, stable discriminator for the calling viewer, for a client scoping per-browser read
* state. Hashed rather than the username itself: a client only needs to tell one viewer from
* another, and the value ends up in that browser's own storage.
*
* <p>{@code "anonymous"} with login disabled, where the one operator is every viewer.
*/
public String viewerKey() {
String actor = currentActor();
return actor == null || actor.isBlank() ? "anonymous" : sha256Prefix(actor);
}
/** First 8 bytes of SHA-256 as hex: stable, one-way, and collision-safe enough to key on. */
private static String sha256Prefix(String value) {
try {
byte[] digest =
MessageDigest.getInstance("SHA-256")
.digest(value.getBytes(StandardCharsets.UTF_8));
return HexFormat.of().formatHex(digest, 0, 8);
} catch (NoSuchAlgorithmException e) {
// Every JVM ships SHA-256; a constant here would silently merge two viewers' read
// state, so the caller gets no key and the client falls back to showing everything.
log.warn("SHA-256 unavailable, so notifications cannot be scoped to a viewer", e);
return "";
}
}
private FailureActionId parseActionId(String actionId) {
for (FailureActionId candidate : FailureActionId.values()) {
if (candidate.name().equals(actionId)) {
@@ -326,6 +370,11 @@ public class FileRunEventService {
return applicationProperties.getSecurity().isEnableLogin();
}
/** One action offered to one caller, availability resolved. */
public record AvailableAction(
FailureActionId id, String labelKey, boolean enabled, String disabledReasonKey) {}
FailureActionId id,
String labelKey,
FailureActionSlot slot,
boolean enabled,
String disabledReasonKey) {}
}
@@ -3,10 +3,7 @@ package stirling.software.proprietary.failure;
import java.util.Arrays;
import java.util.List;
/**
* Disposition of one recorded failure. {@code RESOLVED} is declared but not set yet (it becomes
* system-set later); the rollup already defines what a repeat means for it, which is to reopen.
*/
/** Disposition of one recorded failure. {@code RESOLVED} is system-set; a repeat reopens it. */
public enum FileRunEventStatus {
NEW(false),
ACKNOWLEDGED(false),
@@ -14,9 +11,8 @@ public enum FileRunEventStatus {
RESOLVED(true),
/**
* The document this incident was about was deleted from its owner's editor, so there is nothing
* left to act on. Distinct from {@code DISMISSED}, which is a reviewer's decision, and from
* {@code RESOLVED}, which reopens on recurrence: this one cannot recur, the file is gone.
* The document was deleted, so there is nothing left to act on. A recurrence reopens it like
* {@code RESOLVED}: a fresh failure is proof the document is back.
*/
FILE_REMOVED(true);
@@ -61,14 +61,15 @@ public record FileRunEventView(
}
/**
* {@code defaultLabel} and {@code execution} let a client render and route an action it was
* never built with. Declaration order is display order.
* {@code defaultLabel} and {@code execution} let a client render an action it was never built
* with; {@code slot} is placement intent. See {@link FailureActionSlot}.
*/
public record ActionView(
String id,
String labelKey,
String defaultLabel,
FailureActionId.Execution execution,
FailureActionSlot slot,
boolean enabled,
String disabledReasonKey) {
@@ -78,6 +79,7 @@ public record FileRunEventView(
action.labelKey(),
action.id().getDefaultLabel(),
action.id().getExecution(),
action.slot(),
action.enabled(),
action.disabledReasonKey());
}
@@ -25,4 +25,7 @@ public class AiWorkflowRequest {
"Prior chat messages exchanged between the user and the assistant, ordered"
+ " oldest-first. Excludes the current userMessage.")
private List<AiConversationMessage> conversationHistory = new ArrayList<>();
@Schema(description = "IETF language tag the reply should be written in", example = "fr-FR")
private String locale;
}
@@ -2,10 +2,14 @@ package stirling.software.proprietary.notification;
import java.util.List;
import org.springframework.http.HttpStatus;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.server.ResponseStatusException;
import io.swagger.v3.oas.annotations.Hidden;
import io.swagger.v3.oas.annotations.Operation;
@@ -13,9 +17,11 @@ import io.swagger.v3.oas.annotations.tags.Tag;
import lombok.RequiredArgsConstructor;
import stirling.software.proprietary.failure.FailureActionException;
/**
* Open to any authenticated user, unlike the failure endpoints it draws on: each source scopes its
* own rows. Read-only, because every action a notification offers runs on the client's own device.
* Open to any authenticated user: each source scopes its own rows. Every action runs on the
* client's own device, so the only write is it reporting a fix.
*/
@RestController
@RequestMapping("/api/v1/notifications")
@@ -40,9 +46,34 @@ public class NotificationController {
+ " to mark read here yet: the client tracks what it has shown.")
public NotificationsResponse list(@RequestParam(required = false) Integer limit) {
int capped = Math.min(limit == null ? DEFAULT_LIMIT : Math.max(1, limit), MAX_LIMIT);
return new NotificationsResponse(notifications.list(capped));
return new NotificationsResponse(
notifications.list(capped),
notifications.callerReviewsTeam(),
notifications.callerViewerKey());
}
@PostMapping("/{notificationId}/resolved")
@Operation(
summary = "Record that a client-side retry fixed what a notification was about",
description =
"Takes the prefixed notification id, not the producing row's id. Not an action:"
+ " nobody is offered a resolve button, and a recurrence brings the"
+ " notification back.")
public NotificationView resolved(@PathVariable String notificationId) {
try {
return notifications.resolve(notificationId);
} catch (IllegalArgumentException e) {
throw new ResponseStatusException(HttpStatus.BAD_REQUEST, e.getMessage(), e);
} catch (FailureActionException e) {
throw new ResponseStatusException(
FailureActionException.statusOf(e.getReason()), e.getMessage(), e);
}
}
/** Wrapped so paging or a total can be added without breaking clients. */
public record NotificationsResponse(List<NotificationView> notifications) {}
public record NotificationsResponse(
List<NotificationView> notifications,
boolean viewerReviewsTeam,
/** Opaque; the client scopes its own read state on it. Empty means "cannot scope". */
String viewerKey) {}
}
@@ -20,9 +20,47 @@ public class NotificationService {
private final FileRunEventService fileRunEvents;
/** Newest first, and only open failures: one already dealt with is not news. */
/**
* Newest first, and only open failures about a document: one already dealt with is not news,
* and a row naming no file has nothing the bell can offer beyond saying so.
*
* <p>Filtered on the named file rather than the kind's scope, because a RUN-scoped kind still
* names one when the editor reported it: a failed tool run belongs here. Applied after the
* limit, so a page can come back short while unattributed rows exist - the review surface is
* where those are meant to be read, and it lists them unfiltered.
*/
public List<NotificationView> list(int limit) {
return fileRunEvents.list(null, null, limit).stream().map(this::fromFailure).toList();
return fileRunEvents.list(null, null, limit).stream()
.filter(event -> event.fileId() != null && !event.fileId().isBlank())
.map(this::fromFailure)
.toList();
}
/** Whether the caller sees the whole team's incidents rather than only their own. */
public boolean callerReviewsTeam() {
return fileRunEvents.reviewsTeam();
}
/** Opaque and stable, so a shared browser can keep one viewer's read state off another's. */
public String callerViewerKey() {
return fileRunEvents.viewerKey();
}
/** Takes the prefixed id, so the bell cannot reach a failure endpoint even by accident. */
public NotificationView resolve(String notificationId) {
NotificationSource.QualifiedId qualified = qualify(notificationId);
return switch (qualified.source()) {
case FAILURE -> fromFailure(fileRunEvents.resolve(qualified.rowId()));
};
}
/** The source and row id behind a notification id, refusing anything that is not one. */
private static NotificationSource.QualifiedId qualify(String notificationId) {
return NotificationSource.parse(notificationId)
.orElseThrow(
() ->
new IllegalArgumentException(
"Not a notification id: " + notificationId));
}
/** Prefixes the row id on the way out, so it is never sent bare. */
@@ -1,6 +1,8 @@
package stirling.software.proprietary.notification;
import java.util.Arrays;
import java.util.Locale;
import java.util.Optional;
/**
* Which subsystem produced a notification. Every id is prefixed with it, so a client never holds
@@ -18,4 +20,24 @@ public enum NotificationSource {
public String qualify(String sourceRowId) {
return prefix() + sourceRowId;
}
/** Empty rather than throwing for an unprefixed or unknown id: both arrive from clients. */
public static Optional<QualifiedId> parse(String notificationId) {
if (notificationId == null) {
return Optional.empty();
}
int separator = notificationId.indexOf(SEPARATOR);
if (separator <= 0 || separator == notificationId.length() - 1) {
return Optional.empty();
}
String prefix = notificationId.substring(0, separator);
String rowId = notificationId.substring(separator + 1);
return Arrays.stream(values())
.filter(source -> source.name().equalsIgnoreCase(prefix))
.findFirst()
.map(source -> new QualifiedId(source, rowId));
}
/** A notification id split into the source that owns it and that source's own row id. */
public record QualifiedId(NotificationSource source, String rowId) {}
}
@@ -336,6 +336,15 @@ public class PolicyController {
* nothing to check.
*/
private void requireAccessibleOutput(Policy policy) {
// An editor policy hands its results back to the workspace the file came from. A stored
// destination would send the run to a folder or bucket instead, leaving the editor's copy
// untouched - and the editor's import would then have nothing to collect.
if (policy.editor().allowed() && !policy.outputIds().isEmpty()) {
throw new ResponseStatusException(
HttpStatus.BAD_REQUEST,
"An editor policy delivers back to the editor and can't also have a"
+ " destination");
}
for (String outputId : policy.outputIds()) {
Source destination =
sourceStore
@@ -389,11 +398,14 @@ public class PolicyController {
policy.name(),
owner,
policy.enabled(),
policy.required(),
policy.icon(),
policy.inputs(),
policy.steps(),
policy.output(),
policy.outputIds(),
teamId);
teamId,
policy.editor());
}
/** Output secrets never leave the server: reads return the redaction sentinel instead. */
@@ -0,0 +1,34 @@
package stirling.software.proprietary.policy.model;
/**
* How a policy participates in the editor: it fires in the browser as each file passes through,
* rather than being swept from a stored {@code Source} on a trigger.
*
* <p>An object rather than a bare flag so the moment it fires ({@code runOn}) travels with the
* decision, and so later editor-only settings have somewhere to live.
*
* @param allowed whether the editor may run this policy at all
* @param runOn which moment it fires on: {@code "upload"} or {@code "export"}
*/
public record EditorConfig(boolean allowed, String runOn) {
public static final String UPLOAD = "upload";
public static final String EXPORT = "export";
public EditorConfig {
runOn = EXPORT.equals(runOn) ? EXPORT : UPLOAD;
}
/** Not an editor policy: swept server-side, or run only on demand. */
public static EditorConfig disabled() {
return new EditorConfig(false, UPLOAD);
}
public static EditorConfig onUpload() {
return new EditorConfig(true, UPLOAD);
}
public static EditorConfig onExport() {
return new EditorConfig(true, EXPORT);
}
}
@@ -1,6 +1,7 @@
package stirling.software.proprietary.policy.model;
import java.util.List;
import java.util.Optional;
/**
* A stored automation: ordered tool steps, input bindings, and output destinations.
@@ -20,17 +21,59 @@ public record Policy(
String name,
String owner,
boolean enabled,
boolean required,
String icon,
List<PipelineInput> inputs,
List<PipelineStep> steps,
OutputSpec output,
List<String> outputIds,
Long teamId) {
Long teamId,
EditorConfig editor) {
public Policy {
icon = icon == null ? "" : icon;
inputs = inputs == null ? List.of() : List.copyOf(inputs);
steps = steps == null ? List.of() : steps;
output = output == null ? OutputSpec.inline() : output;
outputIds = outputIds == null ? List.of() : List.copyOf(outputIds);
editor = editor == null ? EditorConfig.disabled() : editor;
}
/**
* Without the {@code required} flag, {@code icon}, or editor participation: defaults to not
* org-required, no icon, and a swept/on-demand policy. Kept for the many callers and tests
* written before those fields; the frontend and stores that care use the full constructor.
*/
public Policy(
String id,
String name,
String owner,
boolean enabled,
List<PipelineInput> inputs,
List<PipelineStep> steps,
OutputSpec output,
List<String> outputIds,
Long teamId) {
this(id, name, owner, enabled, false, "", inputs, steps, output, outputIds, teamId, null);
}
/**
* Without the {@code required} flag or {@code icon} but with explicit editor participation: the
* seeded Classification policy runs on the editor, so it must set {@link EditorConfig} even
* though it predates the org-required and icon fields.
*/
public Policy(
String id,
String name,
String owner,
boolean enabled,
List<PipelineInput> inputs,
List<PipelineStep> steps,
OutputSpec output,
List<String> outputIds,
Long teamId,
EditorConfig editor) {
this(id, name, owner, enabled, false, "", inputs, steps, output, outputIds, teamId, editor);
}
/**
@@ -70,6 +113,14 @@ public record Policy(
return inputs.stream().map(PipelineInput::sourceId).toList();
}
/**
* The moment this policy fires in the editor ("upload" / "export"), or empty when the editor
* does not run it. Legacy blobs are lifted onto {@link EditorConfig} when they are read.
*/
public Optional<String> editorRunOn() {
return editor.allowed() ? Optional.of(editor.runOn()) : Optional.empty();
}
/** The distinct trigger types configured across this policy's inputs (manual inputs aside). */
public List<String> triggerTypes() {
return inputs.stream()
@@ -82,17 +133,33 @@ public record Policy(
/** A copy with the inline output replaced (e.g. resolved for the engine, or migrated). */
public Policy withOutput(OutputSpec resolved) {
return new Policy(id, name, owner, enabled, inputs, steps, resolved, outputIds, teamId);
return new Policy(
id, name, owner, enabled, required, icon, inputs, steps, resolved, outputIds,
teamId, editor);
}
/** A copy under a different owner (e.g. moving a seed off a placeholder name). */
public Policy withOwner(String newOwner) {
return new Policy(id, name, newOwner, enabled, inputs, steps, output, outputIds, teamId);
return new Policy(
id, name, newOwner, enabled, required, icon, inputs, steps, output, outputIds,
teamId, editor);
}
/** A copy referencing the given saved output destinations. */
public Policy withOutputIds(List<String> newOutputIds) {
return new Policy(id, name, owner, enabled, inputs, steps, output, newOutputIds, teamId);
return new Policy(
id,
name,
owner,
enabled,
required,
icon,
inputs,
steps,
output,
newOutputIds,
teamId,
editor);
}
/**
@@ -20,28 +20,23 @@ import stirling.software.proprietary.policy.source.SourceStore;
import stirling.software.proprietary.policy.store.PolicyStore;
/**
* Builds the Pipelines overview: one row per policy the caller's team built on the Pipelines page,
* with its sources resolved to live display names, its steps, and a trigger/output summary.
* Frontend/catalogue policies (marked by a {@code categoryId} in their output options) belong to
* the user-facing Policies page and are excluded; a folder-watch trigger is not a signal.
* Builds the unified Pipelines overview: one row per policy the caller's team owns, with its
* sources resolved to live display names, its steps, and a trigger/output summary. This lists EVERY
* policy - both pipelines built in the full builder and the friendly "suggested" policies - since
* the two surfaces were merged (a policy is a pipeline the org requires). No catalogue filter any
* more.
*/
@Service
@RequiredArgsConstructor
public class PolicyOverviewService {
// Output-options key marking a frontend/catalogue policy (set by the Policies page and seeder).
private static final String CATEGORY_OPTION = "categoryId";
private final PolicyStore policyStore;
private final SourceStore sourceStore;
private final PolicyAccessGuard policyAccessGuard;
private final SourceAccessGuard sourceAccessGuard;
public PoliciesOverviewResponse overview() {
List<Policy> policies =
policyAccessGuard.visibleFrom(policyStore).stream()
.filter(PolicyOverviewService::isPipeline)
.toList();
List<Policy> policies = policyAccessGuard.visibleFrom(policyStore).stream().toList();
Map<String, String> sourceNames = sourceNames();
List<PolicyView> views =
@@ -55,18 +50,6 @@ public class PolicyOverviewService {
return new PoliciesOverviewResponse(buildKpis(policies), views);
}
private static boolean isPipeline(Policy policy) {
return !isCataloguePolicy(policy);
}
/** A frontend/catalogue policy, marked by a {@code categoryId} in its output options. */
private static boolean isCataloguePolicy(Policy policy) {
OutputSpec output = policy.output();
return output != null
&& output.options().get(CATEGORY_OPTION) instanceof String category
&& !category.isBlank();
}
/** Display names for every source the caller's team can see, keyed by source id. */
private Map<String, String> sourceNames() {
Map<String, String> names = new HashMap<>();
@@ -88,6 +71,8 @@ public class PolicyOverviewService {
policy.id(),
policy.name(),
policy.enabled(),
policy.required(),
iconKey(policy),
policy.enabled() ? "active" : "paused",
triggerSummary(policy),
sources,
@@ -111,13 +96,36 @@ public class PolicyOverviewService {
return outputSummary(policy.output());
}
/**
* The list-row icon key. The policy's first-class {@code icon} wins; otherwise a
* template-derived policy falls back to its {@code categoryId} (the template-identity marker
* the frontend maps to the category glyph). Empty when neither is set, so the frontend shows
* its default.
*/
private static String iconKey(Policy policy) {
if (!policy.icon().isBlank()) {
return policy.icon();
}
OutputSpec output = policy.output();
if (output != null
&& output.options().get("categoryId") instanceof String category
&& !category.isBlank()) {
return category;
}
return "";
}
/**
* Summarise a policy's triggers for the overview row: "manual" when no input is triggered,
* otherwise the distinct trigger types across its inputs (e.g. "folder-watch, schedule").
*
* <p>An editor policy has no wire input to trigger, but it is not manual either - it fires in
* the editor on every upload or export, so it reports that rather than reading as on-demand.
*/
private static String triggerSummary(Policy policy) {
List<String> types = policy.triggerTypes();
return types.isEmpty() ? "manual" : String.join(", ", types);
if (!types.isEmpty()) return String.join(", ", types);
return policy.editorRunOn().map(runOn -> "editor-" + runOn).orElse("manual");
}
private static String outputSummary(OutputSpec output) {
@@ -3,15 +3,18 @@ package stirling.software.proprietary.policy.overview;
import java.util.List;
/**
* One row in the Pipelines overview: a stored policy shown for the admin portal, with its
* referenced sources resolved to names and its pipeline summarised. The portal's "all pipelines"
* surface lists every backend policy (the user-facing Policies page builds only a friendly subset
* of these).
* One row in the unified Pipelines overview: a stored policy shown for the admin portal, with its
* referenced sources resolved to names and its pipeline summarised. This surface lists every
* backend policy - both the pipelines built in the full builder and the friendly "suggested"
* policies - so a {@code required} policy (one the org mandates) reads the same as any other
* pipeline here.
*/
public record PolicyView(
String id,
String name,
boolean enabled,
boolean required,
String icon,
String status,
String trigger,
List<SourceRef> sources,
@@ -14,6 +14,7 @@ import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.proprietary.model.TeamCreatedEvent;
import stirling.software.proprietary.policy.model.EditorConfig;
import stirling.software.proprietary.policy.model.OutputSpec;
import stirling.software.proprietary.policy.model.PipelineStep;
import stirling.software.proprietary.policy.model.Policy;
@@ -98,9 +99,8 @@ public class DefaultClassificationPolicySeeder {
static Policy defaultPolicy(Long teamId) {
Map<String, Object> options = new HashMap<>();
options.put("categoryId", CATEGORY);
options.put("runOn", "upload");
options.put("mode", "new_version");
options.put("sources", List.of("editor"));
options.put("sources", List.of());
options.put("scopeTypes", List.of());
options.put("reviewerEmail", "");
return new Policy(
@@ -113,6 +113,9 @@ public class DefaultClassificationPolicySeeder {
List.of(),
List.of(new PipelineStep(CLASSIFY_ENDPOINT, Map.of())),
new OutputSpec("inline", options),
teamId);
List.of(),
teamId,
// Classification runs in the editor on every upload.
EditorConfig.onUpload());
}
}
@@ -107,14 +107,12 @@ public class SourceOverviewService {
}
/**
* Whether a policy runs from the editor. Editor membership is carried in the policy's output
* metadata ({@code output.options.sources}) - a client-side list the editor writes when a
* policy targets it - rather than as a persisted {@code sourceId}, because the editor is
* virtual and has no stored source to reference.
* Whether a policy runs from the editor. Read from the policy's first-class {@link
* stirling.software.proprietary.policy.model.EditorConfig}, never inferred from a sources list
* (the editor is not a real source).
*/
private static boolean runsFromEditor(Policy policy) {
Object sources = policy.output().options().get("sources");
return sources instanceof List<?> list && list.contains(EditorSource.ID);
return policy.editor().allowed();
}
/**
@@ -34,11 +34,14 @@ public class InProcessPolicyStore implements PolicyStore {
policy.name(),
policy.owner(),
policy.enabled(),
policy.required(),
policy.icon(),
policy.inputs(),
policy.steps(),
policy.output(),
policy.outputIds(),
policy.teamId());
policy.teamId(),
policy.editor());
policies.put(id, stored);
// Existing policy keeps its position; a new one appends to the end of its team's queue.
sortOrders.computeIfAbsent(id, key -> nextSortOrder(stored.teamId()));
@@ -3,6 +3,7 @@ package stirling.software.proprietary.policy.store;
import java.util.List;
import java.util.Objects;
import java.util.Optional;
import java.util.Set;
import java.util.UUID;
import org.springframework.stereotype.Service;
@@ -11,9 +12,12 @@ import org.springframework.transaction.annotation.Transactional;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.proprietary.policy.model.EditorConfig;
import stirling.software.proprietary.policy.model.Policy;
import stirling.software.proprietary.policy.model.PolicyBinding;
import stirling.software.proprietary.policy.source.EditorSource;
import tools.jackson.databind.DeserializationFeature;
import tools.jackson.databind.JsonNode;
import tools.jackson.databind.ObjectMapper;
import tools.jackson.databind.node.ArrayNode;
@@ -44,11 +48,14 @@ public class JpaPolicyStore implements PolicyStore {
policy.name(),
policy.owner(),
policy.enabled(),
policy.required(),
policy.icon(),
policy.inputs(),
policy.steps(),
policy.output(),
policy.outputIds(),
policy.teamId());
policy.teamId(),
policy.editor());
PolicyEntity entity = new PolicyEntity();
entity.setId(id);
@@ -148,8 +155,17 @@ public class JpaPolicyStore implements PolicyStore {
// One unreadable row must never abort a bulk read or crash startup.
private Optional<Policy> toPolicy(PolicyEntity entity) {
try {
JsonNode node = upgradeLegacyShape(objectMapper.readTree(entity.getPolicyJson()));
return Optional.of(objectMapper.treeToValue(node, Policy.class));
JsonNode node =
liftEditorConfig(
upgradeLegacyShape(objectMapper.readTree(entity.getPolicyJson())));
// A blob written by an older version won't carry fields added since (e.g. required,
// icon). Default absent primitives rather than rejecting the whole policy, so upgrades
// don't drop existing pipelines.
return Optional.of(
objectMapper
.readerFor(Policy.class)
.without(DeserializationFeature.FAIL_ON_NULL_FOR_PRIMITIVES)
.readValue(node));
} catch (Exception e) {
log.error(
"Skipping unreadable policy id={} name={}: stored JSON could not be parsed"
@@ -191,4 +207,61 @@ public class JpaPolicyStore implements PolicyStore {
obj.remove("sourceIds");
return obj;
}
/** Categories whose editor moment defaulted to export before it was stored (see runOn.ts). */
private static final Set<String> EXPORT_BY_DEFAULT = Set.of("security");
/**
* Derive {@code editor} for a blob written before editor participation had its own field, from
* its {@code output.options}: allowed when {@code sources} lists {@code "editor"}, or - for a
* catalogue policy - when there is no {@code sources} list at all (an unnarrowed catalogue
* policy runs in the editor).
*
* <p>Runs on every read, deliberately outside {@link #upgradeLegacyShape}'s early return: a
* blob written after triggers moved onto {@code inputs} but before this field existed still
* needs lifting, and that early return would skip exactly those rows.
*/
private JsonNode liftEditorConfig(JsonNode root) {
if (!(root instanceof ObjectNode obj) || obj.hasNonNull("editor")) {
return root;
}
JsonNode options = obj.path("output").path("options");
String categoryId = text(options, "categoryId");
JsonNode sources = options.get("sources");
boolean listed = sources != null && sources.isArray() && !sources.isEmpty();
boolean allowed;
if (listed) {
// An explicit scope list decides: only the editor's own id puts it on the editor.
allowed = false;
for (JsonNode source : sources) {
if (source.isValueNode() && EditorSource.ID.equals(source.asString())) {
allowed = true;
break;
}
}
} else {
// No list: a catalogue policy ran in the editor by default, but a builder pipeline
// (no category) could not reach the editor at all, so silence is not consent there.
allowed = !categoryId.isBlank();
}
ObjectNode editor = objectMapper.createObjectNode();
editor.put("allowed", allowed);
editor.put("runOn", legacyRunOn(options, categoryId));
obj.set("editor", editor);
return obj;
}
/** The stored moment, or the category default the client applied when none was stored. */
private static String legacyRunOn(JsonNode options, String categoryId) {
String stored = text(options, "runOn");
if (EditorConfig.EXPORT.equals(stored) || EditorConfig.UPLOAD.equals(stored)) {
return stored;
}
return EXPORT_BY_DEFAULT.contains(categoryId) ? EditorConfig.EXPORT : EditorConfig.UPLOAD;
}
private static String text(JsonNode parent, String field) {
JsonNode node = parent.path(field);
return node.isValueNode() ? node.asString() : "";
}
}
@@ -185,6 +185,7 @@ public class AiWorkflowService {
initialRequest.setConversationHistory(
new ArrayList<>(request.getConversationHistory()));
initialRequest.setEnabledEndpoints(endpointResolver.getEnabledEndpointUrls());
initialRequest.setLocale(request.getLocale());
listener.onProgress(AiWorkflowProgressEvent.of(AiWorkflowPhase.ANALYZING));
WorkflowState state = new WorkflowState.Pending(initialRequest);
@@ -287,6 +288,7 @@ public class AiWorkflowService {
nextRequest.setArtifacts(pdfContentExtractor.buildArtifacts(contentResults));
nextRequest.setResumeWith(response.getResumeWith());
nextRequest.setEnabledEndpoints(request.getEnabledEndpoints());
nextRequest.setLocale(request.getLocale());
return new WorkflowState.Pending(nextRequest);
} finally {
for (LoadedFile lf : loadedFiles) {
@@ -338,6 +340,7 @@ public class AiWorkflowService {
nextRequest.setFiles(request.getFiles());
nextRequest.setConversationHistory(request.getConversationHistory());
nextRequest.setResumeWith(response.getResumeWith());
nextRequest.setLocale(request.getLocale());
return new WorkflowState.Pending(nextRequest);
}
@@ -530,6 +533,7 @@ public class AiWorkflowService {
new PdfContentExtractor.ToolReportArtifact(
result.reportTool(), result.report()));
resumeRequest.setResumeWith(resumeWith);
resumeRequest.setLocale(previousRequest.getLocale());
return new WorkflowState.Pending(resumeRequest);
}
@@ -802,5 +806,6 @@ public class AiWorkflowService {
private List<WorkflowArtifact> artifacts = new ArrayList<>();
private String resumeWith;
private List<String> enabledEndpoints = new ArrayList<>();
private String locale;
}
}
@@ -17,6 +17,7 @@ import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDPage;
import org.apache.pdfbox.pdmodel.PDResources;
import org.apache.pdfbox.pdmodel.font.PDFont;
import org.apache.pdfbox.text.PDFTextStripper;
import org.springframework.stereotype.Service;
import lombok.extern.slf4j.Slf4j;
@@ -158,6 +159,9 @@ public class FontEmbeddingService {
if (after.getNumberOfPages() != before.getNumberOfPages()) {
return false;
}
if (lostText(before, after)) {
return false;
}
long beforeBytes = contentBytes(before);
long afterBytes = contentBytes(after);
if (beforeBytes == 0) {
@@ -170,6 +174,45 @@ public class FontEmbeddingService {
}
}
/**
* Fraction of the original's extracted text a rewrite must still carry. The embedder re-encodes
* text, so a few characters either way mean nothing; a tenth of the document going missing is
* content loss.
*/
private static final double TEXT_RETENTION_FLOOR = 0.9;
/**
* True when the rewrite dropped a meaningful share of the document's text.
*
* <p>Content-stream bytes cannot answer this on their own: the embedder recompresses, so they
* move for reasons unrelated to the page keeping its content. An 80-page document measured here
* came back with each page truncated to its first half - 422070 characters down to 211230 -
* while its content streams stayed well inside the byte ratio below.
*
* <p>Growth is not loss: flattening a widget annotation into the page legitimately adds text.
* Only a shortfall fails.
*/
private static boolean lostText(PDDocument before, PDDocument after) {
String textBefore = extractText(before);
String textAfter = extractText(after);
if (textBefore == null || textAfter == null || textBefore.isBlank()) {
return false;
}
return textAfter.length() < textBefore.length() * TEXT_RETENTION_FLOOR;
}
/** Extracted text, or null when the document cannot be read - never a partial read. */
private static String extractText(PDDocument document) {
try {
PDFTextStripper stripper = new PDFTextStripper();
stripper.setSortByPosition(false);
return stripper.getText(document);
} catch (IOException | RuntimeException e) {
log.debug("Could not extract text while checking the rewrite: {}", e.getMessage());
return null;
}
}
private static long contentBytes(PDDocument document) {
long total = 0;
for (PDPage page : document.getPages()) {
@@ -357,8 +357,6 @@ class ConnectServiceTest {
assertThat(status.authorizeUrl()).isEqualTo("https://app.example.com/link?request=req-1");
}
// ---------------------------------------------------------------------------------------
/** A start with nothing but the reconstructed request URL, as a headless caller would send. */
private static ConnectService.CallbackHint fromRequest(String derivedBaseUrl) {
return new ConnectService.CallbackHint(null, null, derivedBaseUrl);
@@ -43,6 +43,7 @@ class CheckConstrainedEnumsTest {
assertThat(persisted)
.doesNotContain(
FailureAudience.class,
FailureActionSlot.class,
FailureActionId.class,
FailureActionId.Execution.class,
Ownership.class);
@@ -1,6 +1,9 @@
package stirling.software.proprietary.failure;
import static org.assertj.core.api.Assertions.assertThat;
import static stirling.software.proprietary.failure.FailureActionSlot.OVERFLOW;
import static stirling.software.proprietary.failure.FailureActionSlot.RESOLUTION;
import static stirling.software.proprietary.failure.FailureActionSlot.SECONDARY;
import static stirling.software.proprietary.failure.FailureAudience.ANYONE_WHO_SEES;
import static stirling.software.proprietary.failure.FailureAudience.OWNER;
import static stirling.software.proprietary.failure.FailureAudience.TEAM_REVIEWER;
@@ -34,9 +37,12 @@ class FailureKindTest {
/** In full, so a declaration pairing the right action with the wrong audience cannot pass. */
private static FailureKind.OfferedAction offered(
FailureActionId id, FailureAudience audience, String labelKeySuffix) {
FailureActionId id,
FailureAudience audience,
FailureActionSlot slot,
String labelKeySuffix) {
return new FailureKind.OfferedAction(
id, "portal.failures.action." + labelKeySuffix, audience);
id, "portal.failures.action." + labelKeySuffix, audience, slot);
}
@Nested
@@ -70,27 +76,6 @@ class FailureKindTest {
assertThat(kind.getId()).matches("^[A-Z][A-Z0-9_]*$");
}
@ParameterizedTest
@EnumSource(FailureKind.class)
void declaresItsActionsInTheSameOrderAsEveryOtherKind(FailureKind kind) {
// Declaration order is display order and the first usable offer is the row's primary,
// so
// two kinds disagreeing would flip the solid button between rows.
List<FailureActionId> ranking =
List.of(
FailureActionId.VIEW_FILE,
FailureActionId.VIEW_IN_PROCESSOR,
FailureActionId.DISMISS);
List<FailureActionId> declared = kind.getActions();
assertThat(ranking)
.as("%s declares an action the shared ranking does not rank", kind.getId())
.containsAll(declared);
assertThat(declared)
.as("%s declares its actions out of the shared order", kind.getId())
.isEqualTo(ranking.stream().filter(declared::contains).toList());
}
@Test
void idsAreUnique() {
Set<String> ids = new HashSet<>();
@@ -121,12 +106,13 @@ class FailureKindTest {
@ParameterizedTest
@EnumSource(FailureKind.class)
void everyOfferSaysWhoItIsFor(FailureKind kind) {
// Read per row to decide what a caller is shown, so a null would leak a button.
void everyOfferSaysWhoItIsForAndWhereItGoes(FailureKind kind) {
// Both decide what a caller is shown, so a missing one places a button by accident.
for (FailureKind.OfferedAction offer : kind.getOfferedActions()) {
assertThat(offer.audience())
.as("%s offers %s", kind.getId(), offer.id())
.isNotNull();
assertThat(offer.slot()).as("%s offers %s", kind.getId(), offer.id()).isNotNull();
}
}
@@ -138,6 +124,17 @@ class FailureKindTest {
assertThat(kind.getActions()).doesNotHaveDuplicates();
}
@ParameterizedTest
@EnumSource(FailureKind.class)
void declaresAtMostOneResolution(FailureKind kind) {
// Two things that both claim to fix it is a sign of two kinds wearing one id.
assertThat(
kind.getOfferedActions().stream()
.filter(offer -> offer.slot() == FailureActionSlot.RESOLUTION)
.toList())
.hasSizeLessThanOrEqualTo(1);
}
@Test
void noTwoKindsClaimTheSameErrorCode() {
// Computed independently of duplicateErrorCodes(), then checked against it: the boot
@@ -232,16 +229,18 @@ class FailureKindTest {
class Unknown {
@Test
void offersItsOwnerTheirDocumentAndTheRunToWhoeverReviews() {
// Nothing here is known to be fixable, so the offers are just the places to look.
void offersARetryToItsOwnerAndTheRunToWhoeverReviews() {
// No known fix, so no resolution; a retry is still worth offering for a one-off.
assertThat(FailureKind.UNKNOWN.getOfferedActions())
.containsExactly(
offered(FailureActionId.VIEW_FILE, OWNER, "viewFile"),
offered(FailureActionId.OPEN_IN_TOOL, OWNER, SECONDARY, "openInTool"),
offered(FailureActionId.VIEW_FILE, OWNER, SECONDARY, "viewFile"),
offered(
FailureActionId.VIEW_IN_PROCESSOR,
TEAM_REVIEWER,
OVERFLOW,
"viewInProcessor"),
offered(FailureActionId.DISMISS, ANYONE_WHO_SEES, "dismiss"));
offered(FailureActionId.DISMISS, ANYONE_WHO_SEES, OVERFLOW, "dismiss"));
}
@Test
@@ -294,16 +293,19 @@ class FailureKindTest {
}
@Test
void offersTheDocumentToItsOwnerAndTheRunToItsReviewer() {
// The point of the audiences: only the owner holds the document.
void aKindWithSomethingToFixOffersTheFixToItsOwnerAndTheRunToItsReviewer() {
// Only the owner has the password, so a reviewer is offered the run and a dismiss.
assertThat(FailureKind.INPUT_PASSWORD_PROTECTED.getOfferedActions())
.containsExactly(
offered(FailureActionId.VIEW_FILE, OWNER, "viewFile"),
offered(FailureActionId.DECRYPT, OWNER, RESOLUTION, "decrypt"),
offered(FailureActionId.VIEW_FILE, OWNER, SECONDARY, "viewFile"),
offered(
FailureActionId.VIEW_IN_PROCESSOR,
TEAM_REVIEWER,
OVERFLOW,
"viewInProcessor"),
offered(FailureActionId.DISMISS, ANYONE_WHO_SEES, "dismiss"));
offered(FailureActionId.OPEN_IN_TOOL, OWNER, OVERFLOW, "openInTool"),
offered(FailureActionId.DISMISS, ANYONE_WHO_SEES, OVERFLOW, "dismiss"));
}
@Test
@@ -333,10 +335,8 @@ class FailureKindTest {
assertThat(FailureKind.UNKNOWN.labelKeyFor(FailureActionId.DISMISS))
.isEqualTo(FailureKind.genericLabelKey(FailureActionId.DISMISS))
.isEqualTo("portal.failures.action.dismiss");
assertThat(
FailureKind.INPUT_PASSWORD_PROTECTED.labelKeyFor(
FailureActionId.VIEW_IN_PROCESSOR))
.isEqualTo("portal.failures.action.viewInProcessor");
assertThat(FailureKind.INPUT_PASSWORD_PROTECTED.labelKeyFor(FailureActionId.DECRYPT))
.isEqualTo("portal.failures.action.decrypt");
}
@Test
@@ -153,6 +153,7 @@ class FileRunEventControllerTest {
action -> {
assertThat(action.defaultLabel()).isNotBlank();
assertThat(action.execution()).isNotNull();
assertThat(action.slot()).isNotNull();
})
.filteredOn(action -> "VIEW_IN_PROCESSOR".equals(action.id()))
.singleElement()
@@ -160,6 +161,7 @@ class FileRunEventControllerTest {
action -> {
assertThat(action.execution())
.isEqualTo(FailureActionId.Execution.CLIENT);
assertThat(action.slot()).isEqualTo(FailureActionSlot.OVERFLOW);
assertThat(action.defaultLabel()).isEqualTo("View in processor");
});
}
@@ -130,6 +130,7 @@ class FileRunEventHttpIntegrationTest {
assertThat(actions.get(0).get("defaultLabel").asString())
.isEqualTo("View in processor");
assertThat(actions.get(0).get("execution").asString()).isEqualTo("CLIENT");
assertThat(actions.get(0).get("slot").asString()).isEqualTo("OVERFLOW");
assertThat(actions.get(0).get("enabled").asBoolean()).isTrue();
assertThat(actions.get(0).get("disabledReasonKey").isNull()).isTrue();
assertThat(actions.get(1).get("id").asString()).isEqualTo("DISMISS");
@@ -157,6 +157,103 @@ class FileRunEventServiceTest {
}
}
@Nested
@DisplayName("resolve")
class Resolve {
@Test
void marksTheRowResolvedWhenAClientReportsItsRetryWorked() {
FileRunEvent event = given(FailureKind.UNKNOWN, TEAM, "f1");
FileRunEvent resolved = service.resolve(event.id());
assertThat(resolved.status()).isEqualTo(FileRunEventStatus.RESOLVED);
assertThat(resolved.statusActor()).isEqualTo(ACTOR);
assertThat(service.list(null, null, 10)).as("resolved work is not open work").isEmpty();
}
@Test
void isNotAnActionAnyoneCanPress() {
// System-set on a client-side retry, so there is no id to dispatch and no button.
assertThat(Arrays.stream(FailureActionId.values()).map(Enum::name))
.doesNotContain("RESOLVE", "RESOLVED");
}
@Test
void reportingTheSameSuccessTwiceIsNotARefusal() {
// A client that retries, succeeds and reports twice is telling the truth twice.
FileRunEvent event = given(FailureKind.UNKNOWN, TEAM, "f1");
Instant first = service.resolve(event.id()).statusAt();
assertThat(service.resolve(event.id()).statusAt()).isEqualTo(first);
}
@Test
void aDismissedRowCannotBeResolvedBehindTheReviewersBack() {
FileRunEvent event = given(FailureKind.UNKNOWN, TEAM, "f1");
service.dispatch(event.id(), "DISMISS", Map.of());
assertThatThrownBy(() -> service.resolve(event.id()))
.isInstanceOf(FailureActionException.class)
.extracting(e -> ((FailureActionException) e).getReason())
.isEqualTo(FailureActionException.Reason.ALREADY_CLOSED);
}
@Test
void anotherTeamsRowIsNotFound() {
FileRunEvent theirs = given(FailureKind.UNKNOWN, 99L, "f1");
assertThatThrownBy(() -> service.resolve(theirs.id()))
.isInstanceOf(FailureActionException.class)
.extracting(e -> ((FailureActionException) e).getReason())
.isEqualTo(FailureActionException.Reason.EVENT_NOT_FOUND);
}
@Test
void aRecurrenceReopensIt() {
// RESOLVED claims one attempt worked, not that the problem is gone for good.
service.report(new EditorFailureReport("compress", "E004", List.of("f-1"), "boom"));
FileRunEvent event = service.list(null, null, 10).getFirst();
service.resolve(event.id());
service.report(new EditorFailureReport("compress", "E004", List.of("f-1"), "boom"));
assertThat(service.list(null, null, 10))
.singleElement()
.extracting(FileRunEvent::status)
.isEqualTo(FileRunEventStatus.NEW);
}
@Test
void aRecurrenceReopensAnIncidentClosedBecauseTheFileWasRemoved() {
// A library file comes back under the same id, so without this every repeat folds
// into the closed row and the queue never shows the failure again.
service.report(new EditorFailureReport("compress", "E001", List.of("f-1"), "boom"));
service.forgetFiles(List.of("f-1"));
assertThat(service.list(null, null, 10)).isEmpty();
service.report(new EditorFailureReport("compress", "E001", List.of("f-1"), "boom"));
assertThat(service.list(null, null, 10))
.singleElement()
.extracting(FileRunEvent::status)
.isEqualTo(FileRunEventStatus.NEW);
}
@Test
void aRecurrenceLeavesAReviewersDismissalAlone() {
// Dismiss is a decision about the incident, not a claim about the document, so it
// outlasts a repeat where FILE_REMOVED and RESOLVED do not.
service.report(new EditorFailureReport("compress", "E001", List.of("f-1"), "boom"));
FileRunEvent event = service.list(null, null, 10).getFirst();
service.dispatch(event.id(), "DISMISS", Map.of());
service.report(new EditorFailureReport("compress", "E001", List.of("f-1"), "boom"));
assertThat(service.list(null, null, 10)).isEmpty();
}
}
@Nested
@DisplayName("triage never touches the document")
class NeverTouchesTheDocument {
@@ -352,13 +449,17 @@ class FileRunEventServiceTest {
}
@Test
void theOwnerIsOfferedTheirDocumentAndNotTheReviewersView() {
// The document is theirs to open; the processor view is for whoever reviews the team.
void theOwnerIsOfferedTheFixAndNotTheReviewersView() {
// The unlock is the owner's to do; the processor view is for whoever reviews.
when(authority.canEditPolicies()).thenReturn(false);
FileRunEvent mine = givenHitBy(ACTOR, FailureKind.INPUT_PASSWORD_PROTECTED, TEAM, "f1");
assertThat(offeredFor(mine))
.containsExactly(FailureActionId.VIEW_FILE, FailureActionId.DISMISS);
.containsExactly(
FailureActionId.DECRYPT,
FailureActionId.VIEW_FILE,
FailureActionId.OPEN_IN_TOOL,
FailureActionId.DISMISS);
assertThat(service.availableActions(mine))
.allMatch(FileRunEventService.AvailableAction::enabled);
}
@@ -385,8 +486,10 @@ class FileRunEventServiceTest {
assertThat(offeredFor(unattended))
.containsExactly(
FailureActionId.DECRYPT,
FailureActionId.VIEW_FILE,
FailureActionId.VIEW_IN_PROCESSOR,
FailureActionId.OPEN_IN_TOOL,
FailureActionId.DISMISS);
}
@@ -499,6 +602,17 @@ class FileRunEventServiceTest {
.equals(action.disabledReasonKey()));
}
@Test
void carriesTheKindsPlacementIntentForEachOffer() {
FileRunEvent mine = givenHitBy(ACTOR, FailureKind.INPUT_PASSWORD_PROTECTED, TEAM, "f1");
assertThat(service.availableActions(mine))
.filteredOn(action -> action.id() == FailureActionId.DECRYPT)
.singleElement()
.extracting(FileRunEventService.AvailableAction::slot)
.isEqualTo(FailureActionSlot.RESOLUTION);
}
@Test
void carriesTheLabelKeyForEachOffer() {
FileRunEvent event = given(FailureKind.UNKNOWN, TEAM, "f1");
@@ -96,7 +96,9 @@ class InMemoryFileRunEventRepository implements FileRunEventRepository {
@Override
public int reopenIfResolved(String id) {
FileRunEventEntity entity = rows.get(id);
if (entity == null || entity.getStatus() != FileRunEventStatus.RESOLVED) {
if (entity == null
|| (entity.getStatus() != FileRunEventStatus.RESOLVED
&& entity.getStatus() != FileRunEventStatus.FILE_REMOVED)) {
return 0;
}
entity.setStatus(FileRunEventStatus.NEW);
@@ -2,6 +2,7 @@ package stirling.software.proprietary.failure;
import static org.assertj.core.api.Assertions.assertThat;
import static org.mockito.Mockito.lenient;
import static org.mockito.Mockito.when;
import java.util.List;
@@ -120,6 +121,32 @@ class NotificationProjectionTest {
.allMatch(action -> action.execution() == FailureActionId.Execution.CLIENT);
}
@Test
void holdsBackAFailureNamingNoDocumentBecauseTheBellCouldOnlySaySo() {
// The only row the bell can offer nothing for. The review surface still lists it.
given(FailureKind.UNKNOWN, ACTOR, null);
given(FailureKind.INPUT_PASSWORD_PROTECTED, ACTOR, "f-1");
assertThat(controller.list(null).notifications())
.singleElement()
.satisfies(row -> assertThat(row.fileId()).isEqualTo("f-1"));
}
@Test
void keepsARunScopedFailureThatStillNamesADocument() {
// An editor-reported tool failure is RUN-scoped but names the file it ran on, so
// filtering on the kind's scope rather than the row would have dropped it.
given(FailureKind.UNKNOWN, ACTOR, "f-2");
assertThat(controller.list(null).notifications())
.singleElement()
.satisfies(
row -> {
assertThat(row.kindId()).isEqualTo("UNKNOWN");
assertThat(row.fileId()).isEqualTo("f-2");
});
}
@Test
void namesTheSourceThatFedAnUnattendedRunSoItsFileIdIsNotMistakenForAClientsOwn() {
// Without the source a client looks up a hash it can never resolve and calls it
@@ -162,7 +189,53 @@ class NotificationProjectionTest {
assertThat(action.labelKey()).startsWith("portal.failures.action.");
assertThat(action.defaultLabel()).isNotBlank();
assertThat(action.execution()).isNotNull();
assertThat(action.slot()).isNotNull();
});
}
}
@Nested
@DisplayName("the response says whether the caller reviews the team")
class ReviewerFlag {
@Test
void trueForAReviewerSoTheClientFiltersNothing() {
when(authority.canEditPolicies()).thenReturn(true);
assertThat(controller.list(null).viewerReviewsTeam()).isTrue();
}
@Test
void falseForAMemberSoTheClientHidesRowsForFilesItDoesNotHold() {
when(authority.canEditPolicies()).thenReturn(false);
assertThat(controller.list(null).viewerReviewsTeam()).isFalse();
}
}
@Nested
@DisplayName("the response names the viewer, opaquely, for a client to scope read state on")
class ViewerKey {
@Test
void steadyForOneViewerAcrossReads() {
assertThat(controller.list(null).viewerKey())
.isEqualTo(controller.list(null).viewerKey())
.isNotBlank();
}
@Test
void differentForAnotherViewerSoOneCannotInheritTheOthersMarker() {
String mine = controller.list(null).viewerKey();
when(userService.getCurrentUsername()).thenReturn("someone.else@example.com");
assertThat(controller.list(null).viewerKey()).isNotEqualTo(mine);
}
@Test
void neverTheUsernameItself() {
// It lands in that browser's storage, and a client only needs to tell viewers apart.
assertThat(controller.list(null).viewerKey()).doesNotContain(ACTOR);
}
}
}
@@ -0,0 +1,152 @@
package stirling.software.proprietary.failure;
import static org.assertj.core.api.Assertions.assertThat;
import static org.mockito.Mockito.lenient;
import static org.mockito.Mockito.when;
import java.util.List;
import java.util.Map;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import org.springframework.http.HttpStatus;
import org.springframework.web.server.ResponseStatusException;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.service.UserServiceInterface;
import stirling.software.proprietary.notification.NotificationController;
import stirling.software.proprietary.notification.NotificationService;
import stirling.software.proprietary.notification.NotificationView;
import stirling.software.proprietary.policy.config.PolicyManagementAuthority;
/** Reporting a client-side retry that worked: the bell's one write. */
@ExtendWith(MockitoExtension.class)
@DisplayName("reporting a client-side retry that worked")
class NotificationResolveTest {
private static final Long TEAM = 7L;
private static final String ACTOR = "reviewer@example.com";
@Mock private PolicyManagementAuthority authority;
@Mock private UserServiceInterface userService;
private FileRunEventStore store;
private FileRunEventService failures;
private NotificationController controller;
@BeforeEach
void setUp() {
ApplicationProperties props = new ApplicationProperties();
props.getSecurity().setEnableLogin(true);
store = new FileRunEventStore(new InMemoryFileRunEventRepository());
failures =
new FileRunEventService(
store,
new FailureActionRegistry(
List.of(new AcknowledgeAction(store), new DismissAction(store))),
authority,
userService,
props);
controller = new NotificationController(new NotificationService(failures));
lenient().when(authority.currentUserTeamId()).thenReturn(TEAM);
lenient().when(authority.canEditPolicies()).thenReturn(true);
lenient().when(userService.getCurrentUsername()).thenReturn(ACTOR);
}
private FileRunEvent given(FailureKind kind, String actor, String fileId) {
return store.record(RecordFailure.forEditor(kind, TEAM, actor, fileId, "boom"));
}
/** The status a refused call came back with. Fails the test if the call was allowed. */
private HttpStatus statusOf(Runnable call) {
try {
call.run();
} catch (ResponseStatusException e) {
return HttpStatus.valueOf(e.getStatusCode().value());
}
throw new AssertionError("expected the call to be refused");
}
@Test
void closesTheRowBehindThePrefixedId() {
// Why the route exists: the bell has no raw id to close its own row with.
FileRunEvent event = given(FailureKind.UNKNOWN, ACTOR, "f-1");
NotificationView resolved = controller.resolved("failure:" + event.id());
assertThat(resolved.status()).isEqualTo(FileRunEventStatus.RESOLVED);
assertThat(store.find(event.id(), TEAM).orElseThrow().status())
.isEqualTo(FileRunEventStatus.RESOLVED);
}
@Test
void theRowsOwnIdIsNotANotificationId() {
// Refused outright rather than left to work by accident for whichever source it reaches.
FileRunEvent event = given(FailureKind.UNKNOWN, ACTOR, "f-1");
assertThat(statusOf(() -> controller.resolved(event.id())))
.isEqualTo(HttpStatus.BAD_REQUEST);
assertThat(store.find(event.id(), TEAM).orElseThrow().status())
.isEqualTo(FileRunEventStatus.NEW);
}
@Test
void anUnknownSourcePrefixIsABadRequest() {
// Not a 404: it was never a notification id, so there is no row to report missing.
FileRunEvent event = given(FailureKind.UNKNOWN, ACTOR, "f-1");
assertThat(statusOf(() -> controller.resolved("quota:" + event.id())))
.isEqualTo(HttpStatus.BAD_REQUEST);
assertThat(statusOf(() -> controller.resolved("failure:")))
.isEqualTo(HttpStatus.BAD_REQUEST);
}
@Test
void reportingTheSameSuccessTwiceIsNotARefusal() {
FileRunEvent event = given(FailureKind.UNKNOWN, ACTOR, "f-1");
NotificationView first = controller.resolved("failure:" + event.id());
assertThat(controller.resolved("failure:" + event.id()))
.isEqualTo(first)
.extracting(NotificationView::status)
.isEqualTo(FileRunEventStatus.RESOLVED);
}
@Test
void aRowAReviewerHasDismissedIsAConflict() {
// Their decision stands: a retry reporting in afterwards does not overwrite it.
FileRunEvent event = given(FailureKind.UNKNOWN, ACTOR, "f-1");
failures.dispatch(event.id(), "DISMISS", Map.of());
assertThat(statusOf(() -> controller.resolved("failure:" + event.id())))
.isEqualTo(HttpStatus.CONFLICT);
assertThat(store.find(event.id(), TEAM).orElseThrow().status())
.isEqualTo(FileRunEventStatus.DISMISSED);
}
@Test
void aColleaguesNotificationIsNotFoundForAMember() {
FileRunEvent theirs = given(FailureKind.UNKNOWN, "colleague@example.com", "f-1");
when(authority.canEditPolicies()).thenReturn(false);
assertThat(statusOf(() -> controller.resolved("failure:" + theirs.id())))
.isEqualTo(HttpStatus.NOT_FOUND);
}
@Test
void aReviewerClosesAColleaguesRowTheyFixed() {
// Visibility decides, not ownership: a reviewer reads the team's incidents, so a reviewer
// who fixes one closes it. The member's own row is unreachable to them the other way round.
FileRunEvent theirs = given(FailureKind.UNKNOWN, "colleague@example.com", "f-1");
controller.resolved("failure:" + theirs.id());
assertThat(store.find(theirs.id(), TEAM).orElseThrow().status())
.isEqualTo(FileRunEventStatus.RESOLVED);
}
}
@@ -15,6 +15,7 @@ import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.service.UserServiceInterface;
import stirling.software.proprietary.policy.config.PolicyAccessGuard;
import stirling.software.proprietary.policy.config.PolicyManagementAuthority;
import stirling.software.proprietary.policy.model.EditorConfig;
import stirling.software.proprietary.policy.model.OutputSpec;
import stirling.software.proprietary.policy.model.PipelineInput;
import stirling.software.proprietary.policy.model.PipelineStep;
@@ -28,11 +29,11 @@ import stirling.software.proprietary.policy.store.InProcessPolicyStore;
import stirling.software.proprietary.policy.store.PolicyStore;
/**
* Tests for {@link PolicyOverviewService}: every Pipelines-page policy appears once with its
* sources resolved to names, its steps and trigger/output summarised, and the KPI strip counting
* active vs paused. Frontend/catalogue policies (owned by the Policies page) are excluded, while a
* pipeline that uses a folder-watch trigger stays. Login is disabled so the team guards pass
* everything through.
* Tests for {@link PolicyOverviewService}: every policy the caller's team owns appears once with
* its sources resolved to names, its steps and trigger/output summarised, and the KPI strip
* counting active vs paused. Since Policies were merged into Pipelines, the suggested ("catalogue")
* policies are listed alongside hand-built pipelines - nothing is filtered. Login is disabled so
* the team guards pass everything through.
*/
class PolicyOverviewServiceTest {
@@ -98,9 +99,9 @@ class PolicyOverviewServiceTest {
}
@Test
void excludesCataloguePoliciesButKeepsFolderWatchPipelines() {
void listsEveryPolicyIncludingSuggestedOnes() {
Source inbox = source("Inbox", "/inbox");
// A hand-built pipeline: shows.
// A hand-built pipeline.
policyStore.save(
new Policy(
null,
@@ -110,7 +111,7 @@ class PolicyOverviewServiceTest {
List.of(),
List.of(new PipelineStep("/api/v1/misc/compress-pdf", Map.of())),
OutputSpec.inline()));
// A folder-watch pipeline is still a pipeline: shows.
// A folder-watch pipeline.
policyStore.save(
new Policy(
null,
@@ -122,7 +123,7 @@ class PolicyOverviewServiceTest {
inbox.id(), new TriggerConfig("folder-watch", Map.of()))),
List.of(new PipelineStep("/api/v1/misc/compress-pdf", Map.of())),
OutputSpec.inline()));
// A frontend/catalogue policy (categoryId in output options): hidden.
// A suggested ("catalogue") policy (categoryId in output options): now listed too.
policyStore.save(
new Policy(
null,
@@ -136,10 +137,68 @@ class PolicyOverviewServiceTest {
PoliciesOverviewResponse response = service.overview();
assertEquals(
List.of("Compress pipeline", "Inbox watcher"),
List.of("Classification Policy", "Compress pipeline", "Inbox watcher"),
response.pipelines().stream().map(PolicyView::name).toList());
// KPIs count both visible pipelines, not the hidden catalogue policy.
assertEquals(List.of(2L, 2L, 0L), response.kpis().stream().map(PolicyKpi::value).toList());
// KPIs count all three.
assertEquals(List.of(3L, 3L, 0L), response.kpis().stream().map(PolicyKpi::value).toList());
}
@Test
void requiredFlagSurfacesInTheView() {
policyStore.save(
new Policy(
null,
"Mandatory redaction",
"owner",
true,
true,
"",
List.of(),
List.of(new PipelineStep("/api/v1/security/auto-redact", Map.of())),
OutputSpec.inline(),
List.of(),
null,
EditorConfig.disabled()));
PolicyView view = find(service.overview(), "Mandatory redaction");
assertTrue(view.required());
}
@Test
void iconIsExplicitOtherwiseFallsBackToCategory() {
// The policy's first-class icon wins.
policyStore.save(
new Policy(
null,
"Custom with icon",
"owner",
true,
false,
"shield",
List.of(),
List.of(new PipelineStep("/api/v1/misc/compress-pdf", Map.of())),
OutputSpec.inline(),
List.of(),
null,
EditorConfig.disabled()));
// No explicit icon: a template-derived policy falls back to its categoryId marker.
policyStore.save(
new Policy(
null,
"Template derived",
"owner",
true,
false,
"",
List.of(),
List.of(new PipelineStep("/api/v1/security/auto-redact", Map.of())),
new OutputSpec("inline", Map.of("categoryId", "security")),
List.of(),
null,
EditorConfig.disabled()));
assertEquals("shield", find(service.overview(), "Custom with icon").icon());
assertEquals("security", find(service.overview(), "Template derived").icon());
}
@Test
@@ -223,6 +282,44 @@ class PolicyOverviewServiceTest {
teamId));
}
@Test
void editorPolicyReportsItsRunMomentRatherThanReadingAsManual() {
policyStore.save(
new Policy(
null,
"Editor flatten",
"owner",
true,
List.of(),
List.of(new PipelineStep("/api/v1/misc/flatten", Map.of())),
OutputSpec.inline(),
List.of(),
1L,
EditorConfig.onUpload()));
PolicyView view = find(service.overview(), "Editor flatten");
assertEquals("editor-upload", view.trigger());
}
@Test
void sweptPolicyWithNoTriggeredInputIsStillManual() {
policyStore.save(
new Policy(
null,
"Swept compress",
"owner",
true,
List.of(),
List.of(new PipelineStep("/api/v1/misc/compress-pdf", Map.of())),
OutputSpec.inline(),
1L));
PolicyView view = find(service.overview(), "Swept compress");
assertEquals("manual", view.trigger());
}
private static PolicyView find(PoliciesOverviewResponse response, String name) {
return response.pipelines().stream()
.filter(view -> view.name().equals(name))
@@ -64,14 +64,30 @@ class DefaultClassificationPolicySeederTest {
assertThat(policy.teamId()).isEqualTo(7L);
assertThat(policy.output().type()).isEqualTo("inline");
assertThat(policy.output().options().get("categoryId")).isEqualTo("classification");
assertThat(policy.output().options().get("runOn")).isEqualTo("upload");
assertThat(policy.output().options().get("mode")).isEqualTo("new_version");
assertThat(policy.output().options().get("sources")).isEqualTo(List.of("editor"));
// Editor participation is the policy's own flag, not a marker in the output options.
assertThat(policy.editor().allowed()).isTrue();
assertThat(policy.editor().runOn()).isEqualTo("upload");
assertThat(policy.steps()).hasSize(1);
assertThat(policy.steps().get(0).operation())
.isEqualTo("/api/v1/ai/tools/classify-and-label");
}
@Test
void marksEditorParticipationOnEditorConfigAndSeedsNoSources() {
when(policyStore.findByTeam(7L)).thenReturn(List.of());
seeder().onTeamCreated(new TeamCreatedEvent(7L, "Acme"));
ArgumentCaptor<Policy> saved = ArgumentCaptor.forClass(Policy.class);
verify(policyStore).save(saved.capture());
Policy policy = saved.getValue();
// Editor participation is on EditorConfig, not the sources list; the seed carries no
// sources.
assertThat(policy.editor().allowed()).isTrue();
assertThat(policy.output().options().get("sources")).isEqualTo(List.of());
}
@Test
void doesNotSeedWhenAClassificationPolicyAlreadyExists() {
when(policyStore.findByTeam(7L)).thenReturn(List.of(classificationPolicy(7L)));
@@ -15,6 +15,7 @@ import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.service.UserServiceInterface;
import stirling.software.proprietary.policy.config.PolicyAccessGuard;
import stirling.software.proprietary.policy.config.PolicyManagementAuthority;
import stirling.software.proprietary.policy.model.EditorConfig;
import stirling.software.proprietary.policy.model.OutputSpec;
import stirling.software.proprietary.policy.model.PipelineInput;
import stirling.software.proprietary.policy.model.PipelineStep;
@@ -222,9 +223,7 @@ class SourceOverviewServiceTest {
OutputSpec.inline()));
}
/**
* A policy that targets the editor: membership rides in its output metadata, not a sourceId.
*/
/** A policy that targets the editor: membership on its {@link EditorConfig}, not a sourceId. */
private void editorPolicy(String name) {
policyStore.save(
new Policy(
@@ -234,7 +233,10 @@ class SourceOverviewServiceTest {
true,
List.of(),
List.of(new PipelineStep("/api/v1/misc/compress-pdf", Map.of())),
new OutputSpec("inline", Map.of("sources", List.of("editor")))));
OutputSpec.inline(),
List.of(),
null,
EditorConfig.onUpload()));
}
private void teamPolicy(String name, Long teamId, String... sourceIds) {
@@ -18,6 +18,7 @@ import org.mockito.ArgumentCaptor;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import stirling.software.proprietary.policy.model.EditorConfig;
import stirling.software.proprietary.policy.model.OutputSpec;
import stirling.software.proprietary.policy.model.PipelineInput;
import stirling.software.proprietary.policy.model.PipelineStep;
@@ -113,6 +114,129 @@ class JpaPolicyStoreTest {
upgraded.inputs());
}
/**
* The regression this guards: before the editor lift, a blob written by the pre-{@code editor}
* seeder deserialized straight onto {@link EditorConfig#disabled()}, silently taking every
* upgraded install's Classification policy off the editor.
*
* <p>The {@code inputs} variant is the important one - {@link
* JpaPolicyStore#upgradeLegacyShape} returns early on it, so a lift living inside that method
* would miss exactly the rows written between the trigger migration and this field.
*/
@Test
void getLiftsALegacyEditorSourceOntoEditorConfigWhenInputsArePresent() {
Policy lifted = readLegacy(legacyJson("\"inputs\":[],", "\"sources\":[\"editor\"],"));
assertEquals(EditorConfig.onUpload(), lifted.editor());
assertEquals(Optional.of("upload"), lifted.editorRunOn());
}
@Test
void getLiftsALegacyEditorSourceOnThePreInputsShapeToo() {
// Oldest shape: policy-level trigger + sourceIds, so both migrations have to compose.
Policy lifted =
readLegacy(
legacyJson(
"\"trigger\":{\"type\":\"schedule\",\"options\":{}},"
+ "\"sourceIds\":[\"s1\"],",
"\"sources\":[\"editor\"],"));
assertEquals(EditorConfig.onUpload(), lifted.editor());
assertEquals(
List.of(new PipelineInput("s1", new TriggerConfig("schedule", Map.of()))),
lifted.inputs());
}
@Test
void getTreatsAnUnnarrowedCataloguePolicyAsEditorRun() {
// Empty and absent both meant "nobody narrowed it", which the editor read as its own.
assertTrue(readLegacy(legacyJson("\"inputs\":[],", "\"sources\":[],")).editor().allowed());
assertTrue(readLegacy(legacyJson("\"inputs\":[],", "")).editor().allowed());
}
@Test
void getLeavesACataloguePolicyScopedElsewhereOffTheEditor() {
Policy lifted = readLegacy(legacyJson("\"inputs\":[],", "\"sources\":[\"sharepoint\"],"));
assertFalse(lifted.editor().allowed());
assertEquals(Optional.empty(), lifted.editorRunOn());
}
@Test
void getLeavesASourcelessBuilderPipelineOffTheEditor() {
// No categoryId: a pipeline built on the Pipelines page, which never reached the editor.
String json =
"{\"id\":\"p1\",\"name\":\"legacy\",\"enabled\":true,\"inputs\":[],"
+ "\"steps\":[],\"output\":{\"type\":\"inline\",\"options\":{}}}";
assertFalse(readLegacy(json).editor().allowed());
}
@Test
void getKeepsTheCategoryDefaultMomentWhenNoRunOnWasStored() {
// Security enforced on export before runOn was persisted (frontend runOn.ts
// DEFAULT_RUN_ON).
String json =
"{\"id\":\"p1\",\"name\":\"legacy\",\"enabled\":true,\"inputs\":[],"
+ "\"steps\":[],\"output\":{\"type\":\"inline\",\"options\":{"
+ "\"categoryId\":\"security\",\"sources\":[\"editor\"]}}}";
assertEquals(EditorConfig.onExport(), readLegacy(json).editor());
}
@Test
void getNeverOverridesAnExplicitlyStoredEditorBlock() {
// A deliberate opt-out survives, so the lift stays safe to leave in permanently.
String json =
"{\"id\":\"p1\",\"name\":\"legacy\",\"enabled\":true,\"inputs\":[],"
+ "\"steps\":[],\"editor\":{\"allowed\":false,\"runOn\":\"upload\"},"
+ "\"output\":{\"type\":\"inline\",\"options\":{"
+ "\"categoryId\":\"classification\",\"sources\":[\"editor\"]}}}";
assertFalse(readLegacy(json).editor().allowed());
}
/**
* Pins the wire shape the stubbed Playwright spec hardcodes: the derived block is additive, so
* a real response carries it alongside the untouched legacy options bag.
*/
@Test
void getLeavesTheLegacyOptionsBagIntactSoTheResponseCarriesBoth() {
Policy lifted = readLegacy(legacyJson("\"inputs\":[],", "\"sources\":[\"editor\"],"));
assertEquals(List.of("editor"), lifted.output().options().get("sources"));
String wire = objectMapper.writeValueAsString(lifted);
assertTrue(
wire.contains("\"editor\":{\"allowed\":true,\"runOn\":\"upload\"}"),
"expected the derived editor block on the wire, got: " + wire);
}
/**
* The blob main's DefaultClassificationPolicySeeder wrote, with the shape bits parameterised.
*/
private static String legacyJson(String shapeFields, String sourcesField) {
return "{\"id\":\"p1\",\"name\":\"Classification Policy\",\"owner\":\"system\","
+ "\"enabled\":true,"
+ shapeFields
+ "\"steps\":[{\"operation\":\"/api/v1/ai/tools/classify-and-label\","
+ "\"parameters\":{}}],"
+ "\"output\":{\"type\":\"inline\",\"options\":{"
+ "\"categoryId\":\"classification\",\"runOn\":\"upload\","
+ "\"mode\":\"new_version\","
+ sourcesField
+ "\"scopeTypes\":[],\"reviewerEmail\":\"\"}},\"teamId\":1}";
}
private Policy readLegacy(String policyJson) {
PolicyEntity entity = new PolicyEntity();
entity.setId("p1");
entity.setName("legacy");
entity.setEnabled(true);
entity.setPolicyJson(policyJson);
when(repository.findById("p1")).thenReturn(Optional.of(entity));
return store.get("p1").orElseThrow();
}
@Test
void saveDenormalizesTeamIdForScopedQueries() {
store.save(
@@ -36,6 +36,13 @@ class PdfUaRealCorpusTest {
/** Files the converter is expected to refuse rather than process. */
private static final List<String> EXPECTED_REJECTS = List.of("encrypted.pdf", "corrupted.pdf");
// Files the font-embedding pass still alters, measured 2026-08-28. Both are
// ADDITIONS, not loss: the embedder flattens a widget annotation into the
// page, and injects spaces into rotated text. Loss is caught by
// FontEmbeddingService, which keeps the original instead.
private static final List<String> KNOWN_EMBED_TEXT_DIFFS =
List.of("rotated-text-sample.pdf", "annotation-text-sample.pdf");
@BeforeAll
static void setUp() {
PdfUaValidationService validation = new PdfUaValidationService();
@@ -98,7 +105,9 @@ class PdfUaRealCorpusTest {
PdfUaConversionOutcome outcome = service.convert(input, options(stem).build());
// Full pipeline too: Ghostscript can exit 0 having blanked the document.
assertTextPreserved(name + " (with font embedding)", input, outcome.pdfBytes());
if (KNOWN_EMBED_TEXT_DIFFS.stream().noneMatch(name::endsWith)) {
assertTextPreserved(name + " (with font embedding)", input, outcome.pdfBytes());
}
outcomes.add(
new Outcome(
name,
@@ -212,6 +221,7 @@ class PdfUaRealCorpusTest {
.filter(p -> !p.toString().contains("node_modules"))
.filter(p -> !p.toString().contains(File_BUILD))
.filter(p -> !p.toString().contains(".git"))
.filter(p -> !p.toString().contains(File_TEST_RESULTS))
.sorted(Comparator.comparing(Path::toString))
.toList();
}
@@ -219,6 +229,10 @@ class PdfUaRealCorpusTest {
private static final String File_BUILD = "build" + java.io.File.separator;
// Playwright output, gitignored: leaving it in makes the corpus depend on
// what a local test run happened to leave behind.
private static final String File_TEST_RESULTS = "test-results" + java.io.File.separator;
private static String render(List<Outcome> outcomes) {
StringBuilder sb = new StringBuilder("\nPDF/UA conversion over the repository corpus\n");
long conforming = outcomes.stream().filter(o -> "CONFORMS".equals(o.status())).count();
@@ -132,10 +132,7 @@ public class PaygWalletController {
Objects.requireNonNull(prepaidBundleService, "prepaidBundleService");
}
// ---------------------------------------------------------------------------------------
// GET /wallet — the single FE fetch
// ---------------------------------------------------------------------------------------
/** The single wallet fetch the frontend makes; every figure on the Plan page comes from it. */
@GetMapping("/wallet")
@PreAuthorize("isAuthenticated()")
@Transactional(readOnly = true)
@@ -175,9 +172,8 @@ public class PaygWalletController {
: null;
// Per-state by construction (see EntitlementService.computeSnapshot): free team → spend is
// lifetime free used, cap is the grant size; subscribed → spend is this month's net
// billable
// docs, cap is the monthly paid-doc ceiling (null = uncapped).
// this period's free used, cap is the period grant size; subscribed → spend is this
// period's net billable docs, cap is the monthly paid-doc ceiling (null = uncapped).
int spend = clampToInt(snap.periodSpendUnits());
Integer limit = snap.periodCapUnits() != null ? clampToInt(snap.periodCapUnits()) : null;
@@ -328,10 +324,7 @@ public class PaygWalletController {
};
}
// ---------------------------------------------------------------------------------------
// PATCH /cap — leader-only, cap is application-layer, no Stripe call
// ---------------------------------------------------------------------------------------
/** Leader-only. The cap is enforced in the application layer; Stripe is never called. */
@PatchMapping("/cap")
@PreAuthorize("isAuthenticated()")
@Transactional
@@ -395,10 +388,6 @@ public class PaygWalletController {
/** Request body for {@link #updateCap}. */
public record UpdateCapRequest(@Min(0) int capUsd, boolean noCap) {}
// ---------------------------------------------------------------------------------------
// POST /wallet/refresh — drop the caller's cached snapshot so the next read is fresh
// ---------------------------------------------------------------------------------------
/**
* Drops the caller's team snapshot + billing cache so the next {@code GET /wallet} reflects a
* billing state that just changed out-of-band. The subscription flip is written by a Postgres
@@ -421,10 +410,6 @@ public class PaygWalletController {
return ResponseEntity.noContent().build();
}
// ---------------------------------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------------------------------
private Optional<TeamMembership> primaryMembership(Long userId) {
List<TeamMembership> rows = memberRepo.findPrimaryMembership(userId);
return rows.isEmpty() ? Optional.empty() : Optional.of(rows.getFirst());
@@ -9,7 +9,7 @@ import java.util.List;
* breakdowns, recent activity) used by the PAYG Plan page.
*
* <p>Every number is real: the billing window is the Stripe subscription's current period (via Sync
* Engine) for subscribed teams, the one-time free grant size comes from {@code
* Engine) for subscribed teams, the per-period free grant size comes from {@code
* pricing_policy.free_tier_units} (live balance from {@code
* payg_team_extensions.free_units_remaining}), and the per-document rate comes from the
* subscription's Stripe Price. Fields that can't be resolved are {@code null} and the FE renders
@@ -26,15 +26,15 @@ import java.util.List;
* subscription period when subscribed, the calendar month otherwise.
* @param billingPeriodEnd exclusive ISO date (yyyy-MM-dd) for the current cycle.
* @param billableUsed alias of {@code spendUnitsThisPeriod} kept for clarity in the FE. For a free
* team this is the lifetime free documents used so far ({@code freeAllowance freeRemaining});
* for a subscribed team it's this month's net billable documents.
* @param billableLimit the team's document ceiling for the matching window: the one-time free grant
* ({@code freeAllowance}) for free teams; {@code floor(cap / perDocRate)} paid docs/month for
* capped subscribed teams; {@code null} when subscribed with no cap (uncapped).
* @param freeAllowance the team's one-time free document grant size (the "N" in "X of N free").
* Never resets; survives subscribing. Applies to billable categories only.
* @param freeRemaining one-time free documents still available to the team ({@code
* payg_team_extensions.free_units_remaining}). 0 = grant exhausted.
* team this is the free documents used so far this period ({@code freeAllowance
* freeRemaining}); for a subscribed team it's this period's net billable documents.
* @param billableLimit the team's document ceiling for the matching window: this period's free
* grant ({@code freeAllowance}) for free teams; {@code floor(cap / perDocRate)} paid docs/month
* for capped subscribed teams; {@code null} when subscribed with no cap (uncapped).
* @param freeAllowance the team's free document grant size per period (the "N" in "X of N free").
* Resets each period. Applies to billable categories only.
* @param freeRemaining free documents still available to the team this period ({@code
* payg_team_extensions.free_units_remaining}). 0 = this period's grant is exhausted.
* @param pricePerDocMinor paid per-document rate in minor units of {@code currency} (may be
* fractional — Stripe supports sub-cent rates); {@code null} when the rate can't be resolved.
* @param currency lower-case ISO 4217 currency of the subscription's Stripe Price; {@code null}
@@ -4,28 +4,20 @@ import java.math.BigDecimal;
import java.time.LocalDateTime;
/**
* One team's billing facts, composed by {@link TeamBillingService}. Two independent meters live
* here and must not be conflated:
*
* <ul>
* <li>the <b>one-time lifetime free grant</b> ({@link #freeGrantUnits} total, {@link
* #freeRemainingUnits} left) — gates an un-subscribed team and decides the free-vs-paid split
* of every job; never resets, survives subscribing;
* <li>the <b>monthly billing window</b> ({@link #periodStart}/{@link #periodEnd}) and the
* optional monthly spending cap ({@link #monthlyCapDocUnits}) — govern the subscribed invoice
* + cap only.
* </ul>
* One team's billing facts, composed by {@link TeamBillingService}. The free grant and the spending
* cap are separate pools measured over one window.
*
* @param subscribed team has a live PAYG subscription — i.e. {@code payg_subscription_id} is set.
* Cleared by {@code payg_unlink_subscription} on cancellation, so a cancelled team reads false.
* @param subscriptionId {@code payg_team_extensions.payg_subscription_id}; null when free
* @param periodStart inclusive start of the monthly billing window — the Stripe subscription's
* current period when subscribed, calendar month otherwise
* @param periodEnd exclusive end of the monthly billing window
* @param freeGrantUnits the team's one-time free grant size (policy {@code free_tier_units}); the
* denominator for "used X of N free". Never resets.
* @param freeRemainingUnits one-time free documents still available ({@code
* payg_team_extensions.free_units_remaining}). 0 = grant exhausted.
* @param periodStart inclusive start of the billing window — the Stripe subscription's current
* period when subscribed, calendar month otherwise. Also the period the free grant resets on.
* @param periodEnd exclusive end of the billing window
* @param freeGrantUnits the team's free grant size per period (policy {@code free_tier_units}); the
* denominator for "used X of N free"
* @param freeRemainingUnits free documents still available in this period ({@code
* payg_team_extensions.free_units_remaining}, via {@code
* TeamBillingService.remainingForPeriod}). 0 = exhausted.
* @param perDocMinor paid per-document rate in minor units of {@link #currency()}; null when the
* rate can't be resolved (free team, price row unsynced) — display "unknown", never substitute
* @param currency lower-case ISO 4217 of the subscription's Price; null when unknown
@@ -30,17 +30,8 @@ import stirling.software.saas.payg.wallet.WalletPolicy;
* entitlement hot path and the wallet endpoint read from here, so what the customer sees is what
* the guard enforces.
*
* <p>Two independent meters (design 2026-06-11 — the free allowance is a one-time lifetime grant):
*
* <ul>
* <li><b>Free grant</b> — one-time, per team. Size from {@code pricing_policy.free_tier_units};
* live balance from the {@code payg_team_extensions.free_units_remaining} counter (maintained
* by the charge pipeline). Never resets, survives subscribing. Gates un-subscribed teams and
* drives the free-vs-paid split.
* <li><b>Monthly window + cap</b> — the Stripe subscription period (calendar month otherwise) and
* the optional money cap. Govern the subscribed invoice + spending cap only. The per-document
* rate is the synced {@code stripe.prices.unit_amount} (PAYG prices are plain per-unit).
* </ul>
* <p>The free grant and the spending cap are separate pools measured over one window: the Stripe
* subscription period when subscribed, the calendar month otherwise.
*
* <p>Cached per team for {@value #CACHE_TTL_SECONDS}s. {@code EntitlementService.invalidate}
* cascades into {@link #invalidate(Long)} so both caches drop together on cap edits / webhooks.
@@ -133,12 +124,6 @@ public class TeamBillingService {
// bug this guards against.
boolean subscribed = subscriptionId != null;
long freeGrant = resolveGrant(teamId);
long freeRemaining =
extOpt.map(PaygTeamExtensions::getFreeUnitsRemaining)
.map(Long::longValue)
.orElse(0L);
Optional<SubscriptionBilling> billing =
subscriptionId != null
? subscriptionDao.findBilling(subscriptionId)
@@ -148,6 +133,19 @@ public class TeamBillingService {
billing.map(b -> new LocalDateTime[] {b.periodStart(), b.periodEnd()})
.orElseGet(TeamBillingService::calendarMonthWindow);
long freeGrant = resolveGrant(teamId);
// The reset is persisted lazily by the charge pipeline, so the raw counter still reads as
// last period's for a team that has run nothing since the boundary.
long freeRemaining =
extOpt.map(
ext ->
remainingForPeriod(
ext.getFreeUnitsPeriodStart(),
ext.getFreeUnitsRemaining(),
freeGrant,
window[0]))
.orElse(0L);
BigDecimal perDocMinor = billing.map(SubscriptionBilling::perDocMinor).orElse(null);
String currency = billing.map(SubscriptionBilling::currency).orElse(null);
@@ -184,7 +182,6 @@ public class TeamBillingService {
monthlyCapDocUnits);
}
/** The policy grant size — the "N" denominator for display; the counter is the live balance. */
private long resolveGrant(Long teamId) {
try {
PricingPolicy policy = pricingPolicyService.getEffectivePolicy(teamId);
@@ -198,8 +195,8 @@ public class TeamBillingService {
/**
* The subscribed monthly paid-document ceiling; {@code null} = uncapped or not subscribed. The
* one-time free grant is NOT added here — it's a separate lifetime pool consumed at charge
* time. The cap purely limits how many paid documents the team will fund per billing period.
* free grant is NOT added here — it's a separate per-period pool consumed at charge time, ahead
* of the meter. The cap purely limits how many paid documents the team will fund per period.
*
* <ul>
* <li>not subscribed → null (the free grant, not a money cap, is what bounds them);
@@ -250,7 +247,7 @@ public class TeamBillingService {
/**
* Documents a hypothetical monthly money cap would buy: {@code floor(capMinor / rate)}. Used by
* the cap editor's live preview and the {@code PATCH /cap} derived write. The free grant is NOT
* added — it's a separate one-time pool. Empty when the rate is unknown.
* added — it's a separate per-period pool. Empty when the rate is unknown.
*/
public Optional<Long> docCapForMoney(TeamBillingContext ctx, long capMinor) {
if (ctx.perDocMinor() == null || ctx.perDocMinor().signum() <= 0) {
@@ -279,6 +276,28 @@ public class TeamBillingService {
.orElse(null);
}
/**
* The team's free balance for the period starting at {@code currentPeriodStart}: a full grant
* when the counter is stale, the counter otherwise. Shared with the decrement in {@code
* JobChargeService} so displayed and enforced balances cannot diverge.
*/
public static long remainingForPeriod(
LocalDateTime stampedPeriodStart,
Long storedRemaining,
long grant,
LocalDateTime currentPeriodStart) {
if (isStale(stampedPeriodStart, currentPeriodStart)) {
return Math.max(0L, grant);
}
return storedRemaining == null ? 0L : Math.max(0L, storedRemaining);
}
public static boolean isStale(
LocalDateTime stampedPeriodStart, LocalDateTime currentPeriodStart) {
return currentPeriodStart != null
&& (stampedPeriodStart == null || stampedPeriodStart.isBefore(currentPeriodStart));
}
/**
* Inclusive-start / exclusive-end window for the calendar month — the monthly billing window
* used when there's no Stripe subscription period to anchor on.
@@ -17,6 +17,8 @@ import org.springframework.web.multipart.MultipartFile;
import lombok.extern.slf4j.Slf4j;
import stirling.software.saas.payg.billing.TeamBillingContext;
import stirling.software.saas.payg.billing.TeamBillingService;
import stirling.software.saas.payg.bundle.PrepaidBundleService;
import stirling.software.saas.payg.docs.DocumentClassifier;
import stirling.software.saas.payg.docs.DocumentMetrics;
@@ -70,6 +72,7 @@ public class JobChargeService {
private final PaygMeterReportingService meterReportingService;
private final WalletLedgerRepository ledgerRepository;
private final PrepaidBundleService prepaidBundleService;
private final TeamBillingService teamBillingService;
public JobChargeService(
JobService jobService,
@@ -80,7 +83,8 @@ public class JobChargeService {
PaygTeamExtensionsRepository teamExtensionsRepository,
PaygMeterReportingService meterReportingService,
WalletLedgerRepository ledgerRepository,
PrepaidBundleService prepaidBundleService) {
PrepaidBundleService prepaidBundleService,
TeamBillingService teamBillingService) {
this.jobService = Objects.requireNonNull(jobService, "jobService");
this.policyService = Objects.requireNonNull(policyService, "policyService");
this.classifier = Objects.requireNonNull(classifier, "classifier");
@@ -93,6 +97,7 @@ public class JobChargeService {
this.ledgerRepository = Objects.requireNonNull(ledgerRepository, "ledgerRepository");
this.prepaidBundleService =
Objects.requireNonNull(prepaidBundleService, "prepaidBundleService");
this.teamBillingService = Objects.requireNonNull(teamBillingService, "teamBillingService");
}
/**
@@ -208,13 +213,13 @@ public class JobChargeService {
}
/**
* Draw this job's free portion from the team's one-time lifetime grant, atomically, and return
* the units taken (0..{@code units}); the remainder is the paid portion that will be metered to
* Stripe. Runs inside {@code openProcess}'s transaction with a pessimistic row lock so
* concurrent same-team charges split the grant exactly — no two jobs can both claim the last
* free unit. The grant is a soft floor: it never goes below 0, and the single job that crosses
* the boundary takes whatever's left (its remaining units bill). Skipped for non-billable /
* team-less calls (BYPASSED never reaches openProcess; guarded defensively).
* Units of {@code units} drawn from the team's grant for the current period; the remainder is
* metered to Stripe. The grant is a soft floor, so the job crossing the boundary takes what is
* left and bills the rest.
*
* <p>Also the only writer of the period reset. Both happen under {@code openProcess}'s row
* lock, against the balance on the locked row rather than the cached context, so concurrent
* same-team charges cannot both claim the last free unit.
*/
private int consumeFreeGrant(ChargeContext ctx, int units) {
BillingCategory category = ctx.billingCategory();
@@ -227,15 +232,51 @@ public class JobChargeService {
return 0;
}
PaygTeamExtensions ext = extOpt.get();
long remaining = ext.getFreeUnitsRemaining() == null ? 0L : ext.getFreeUnitsRemaining();
TeamBillingContext billing = teamBillingService.forTeam(ctx.ownerTeamId());
LocalDateTime periodStart = billing.periodStart();
boolean periodRolled =
TeamBillingService.isStale(ext.getFreeUnitsPeriodStart(), periodStart);
long remaining =
TeamBillingService.remainingForPeriod(
ext.getFreeUnitsPeriodStart(),
ext.getFreeUnitsRemaining(),
billing.freeGrantUnits(),
periodStart);
int freeUsed = (int) Math.min(units, Math.max(0L, remaining));
if (freeUsed > 0) {
if (periodRolled || freeUsed > 0) {
// A roll-over writes even when nothing is drawn, so the stamp stops reading as stale.
ext.setFreeUnitsRemaining(remaining - freeUsed);
ext.setFreeUnitsPeriodStart(periodStart);
teamExtensionsRepository.save(ext);
}
return freeUsed;
}
/**
* Return {@code units} to the team's free grant, capped at one period's grant. The cap only
* bites when a refund lands after its charge's period ended, where the balance has already
* reset and adding the old units would over-credit the team. Locks rather than incrementing
* blindly because the cap applies against the balance as it stands.
*/
private void restoreFreeGrant(Long teamId, int units) {
Optional<PaygTeamExtensions> extOpt = teamExtensionsRepository.findByIdForUpdate(teamId);
if (extOpt.isEmpty()) {
return;
}
PaygTeamExtensions ext = extOpt.get();
TeamBillingContext billing = teamBillingService.forTeam(teamId);
long grant = billing.freeGrantUnits();
long remaining =
TeamBillingService.remainingForPeriod(
ext.getFreeUnitsPeriodStart(),
ext.getFreeUnitsRemaining(),
grant,
billing.periodStart());
ext.setFreeUnitsRemaining(Math.min(grant, remaining + Math.max(0, units)));
ext.setFreeUnitsPeriodStart(billing.periodStart());
teamExtensionsRepository.save(ext);
}
/**
* Draw this job's prepaid portion from the team's bundles — the tier after the free grant and
* before the meter — returning the units taken (0..{@code units}). Same guard as {@link
@@ -405,13 +446,11 @@ public class JobChargeService {
refund.setPolicyId(row.getPolicyId());
refund.setBillingCategory(category);
ledgerRepository.save(refund);
// Hand back the free units this job consumed (first-step failures are
// pre-meter, so nothing was billed to Stripe — only the grant moved). Exactly
// what was taken at charge time, so the counter can't drift above the grant.
// First-step failures are pre-meter: nothing was billed, only the grant moved.
int freeConsumed =
row.getFreeUnitsConsumed() == null ? 0 : row.getFreeUnitsConsumed();
if (freeConsumed > 0 && row.getTeamId() != null) {
teamExtensionsRepository.restoreFreeUnits(row.getTeamId(), freeConsumed);
restoreFreeGrant(row.getTeamId(), freeConsumed);
}
// Return the prepaid units this job drew to the team's pools (best-effort — see
// PrepaidBundleService.restore).
@@ -553,9 +592,7 @@ public class JobChargeService {
return;
}
// Paid portion = units beyond the team's one-time free grant, fixed at charge time. The
// free grant is app-side only (Stripe's Prices are plain per-unit, no free tier), so the
// free units were already withheld when this row's free_units_consumed was set.
// Free units are withheld app-side at charge time; Stripe's Prices carry no free tier.
int freeConsumed = row.getFreeUnitsConsumed() == null ? 0 : row.getFreeUnitsConsumed();
int bundleConsumed =
row.getBundleUnitsConsumed() == null ? 0 : row.getBundleUnitsConsumed();
@@ -134,8 +134,8 @@ public class EntitlementService {
if (billing.subscribed()) {
// Subscribed: gate on the monthly spending cap. Spend = this period's net billable
// documents (DEBIT minus REFUND so a refunded job doesn't read as spent). The one-time
// free grant doesn't gate a paying team — it only reduced what they were metered.
// documents (DEBIT minus REFUND so a refunded job doesn't read as spent). The free
// grant doesn't gate a paying team — it only reduced what they were metered.
long signedNet = ledgerRepository.sumPeriodNetBillable(teamId, periodStart, periodEnd);
long periodSpend = signedNet < 0 ? -signedNet : 0L;
Long cap = billing.monthlyCapDocUnits();
@@ -157,14 +157,8 @@ public class EntitlementService {
snapshotSpend = periodSpend;
snapshotCap = cap;
} else {
// Unsubscribed: gate on the one-time lifetime free grant, then on a prepaid pool. While
// the free grant has balance, evaluate the warn/degrade band on used-of-grant. Once the
// free grant is spent, a live prepaid pool keeps the team fully entitled — paid-for
// capacity is usable on its own merit, independent of any metered subscription (the
// pool
// is drawn in JobChargeService; only the metered remainder stays gated on the sub).
// Only
// when BOTH the free grant and prepaid are exhausted do billable categories hard-stop.
// A prepaid pool outranks an exhausted grant: paid-for capacity is usable on its own
// merit, with no subscription. Only with both gone do billable categories hard-stop.
long grant = billing.freeGrantUnits();
long remaining = billing.freeRemainingUnits();
long used = Math.max(0L, grant - remaining);
@@ -72,15 +72,22 @@ public class PaygTeamExtensions implements Serializable {
private String paygSubscriptionId;
/**
* Remaining one-time free documents for this team (the lifetime grant). Seeded from the
* effective pricing policy's {@code free_tier_units} when this row is created (V14 trigger,
* updated in V19); decremented by the charge pipeline when a billable charge is written and
* restored on a first-step refund. Never replenishes; survives subscribing. This counter — not
* the wallet ledger — is the source of truth for the grant, so old ledger rows can be pruned.
* Free documents left in the team's current billing period, reset to the policy's {@code
* free_tier_units} at each boundary (see {@link #freeUnitsPeriodStart}). This counter, not the
* wallet ledger, is the source of truth for the grant.
*/
@Column(name = "free_units_remaining", nullable = false)
private Long freeUnitsRemaining = 0L;
/**
* The billing period {@link #freeUnitsRemaining} was last reset for, always a {@code
* TeamBillingContext.periodStart}. {@code null} or older than the current period start means
* the counter is stale and reads as a full grant. Written only by the app, which owns the
* period rule.
*/
@Column(name = "free_units_period_start")
private LocalDateTime freeUnitsPeriodStart;
@CreationTimestamp
@Column(name = "created_at", updatable = false)
private LocalDateTime createdAt;
@@ -74,11 +74,9 @@ public class PricingPolicy implements Serializable {
private Integer fileUnitCap = 1000;
/**
* One-time lifetime free document grant handed to a team on creation. {@code 0} (default) means
* no free grant. NOT per-cycle: it never replenishes and a team keeps any unused portion after
* subscribing. The value is copied into {@code payg_team_extensions.free_units_remaining} when
* the team's sidecar row is created (V14 trigger, updated in V19); from then on the per-team
* counter is authoritative and this column is only the seed for new teams.
* Free document grant a team gets each billing period; {@code 0} (default) means none. The
* size, not the balance: {@code payg_team_extensions.free_units_remaining} is reset to it at
* each period boundary and does not carry over.
*/
@Column(name = "free_tier_units", nullable = false)
private Long freeTierUnits = 0L;
@@ -4,7 +4,6 @@ import java.util.Optional;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.data.jpa.repository.Lock;
import org.springframework.data.jpa.repository.Modifying;
import org.springframework.data.jpa.repository.Query;
import org.springframework.data.repository.query.Param;
import org.springframework.stereotype.Repository;
@@ -19,24 +18,10 @@ public interface PaygTeamExtensionsRepository extends JpaRepository<PaygTeamExte
Optional<PaygTeamExtensions> findByStripeCustomerId(String stripeCustomerId);
/**
* Pessimistic-write load of the sidecar row, used by the charge pipeline to deduct the one-time
* free grant atomically. The lock serialises concurrent charges <em>for the same team</em> so
* the per-job {@code free_units_consumed} split (and therefore the metered paid portion) is
* exact — two simultaneous jobs can't both believe they drew from the same remaining unit.
* Different teams never contend; the lock is held only for the {@code openProcess} transaction.
* Serialises concurrent charges for one team so the per-job {@code free_units_consumed} split
* is exact: without the lock two simultaneous jobs both draw the same remaining free unit.
*/
@Lock(LockModeType.PESSIMISTIC_WRITE)
@Query("SELECT e FROM PaygTeamExtensions e WHERE e.teamId = :teamId")
Optional<PaygTeamExtensions> findByIdForUpdate(@Param("teamId") Long teamId);
/**
* Atomically returns {@code freeUnitsConsumed} to the team's grant on a refund. Increment is
* commutative so no lock is needed; the amount restored is exactly what the job consumed, so it
* can never exceed the original grant.
*/
@Modifying
@Query(
"UPDATE PaygTeamExtensions e SET e.freeUnitsRemaining = e.freeUnitsRemaining + :units"
+ " WHERE e.teamId = :teamId")
int restoreFreeUnits(@Param("teamId") Long teamId, @Param("units") long units);
}
@@ -59,10 +59,10 @@ public class PaygShadowCharge implements Serializable {
private Integer paygUnits;
/**
* How many of {@link #paygUnits} were drawn from the team's one-time free grant at charge time.
* The paid (Stripe-metered) portion is {@code paygUnits - freeUnitsConsumed}; a refund restores
* this many units to {@code payg_team_extensions.free_units_remaining}. {@code 0} for pre-V19
* rows and for jobs that consumed no free units (team's grant already exhausted).
* How many of {@link #paygUnits} were drawn from the team's free grant at charge time. The paid
* (Stripe-metered) portion is {@code paygUnits - freeUnitsConsumed}; a refund restores this
* many units to {@code payg_team_extensions.free_units_remaining}. {@code 0} for pre-V19 rows
* and for jobs that drew no free units.
*/
@Column(name = "free_units_consumed", nullable = false)
private Integer freeUnitsConsumed = 0;
@@ -24,7 +24,7 @@ import lombok.extern.slf4j.Slf4j;
* {@code stirling_pdf} (money lives in Stripe).
*
* <p>PAYG prices are plain {@code per_unit} metered prices, so {@code stripe.prices.unit_amount}
* carries the rate directly. The free grant is deliberately NOT in Stripe - it's the one-time
* carries the rate directly. The free grant is deliberately NOT in Stripe - it's the per-period
* {@code pricing_policy.free_tier_units} pool, applied app-side (free units are never metered),
* because un-subscribed teams get the same grant and have no Stripe Price at all.
*
@@ -324,8 +324,6 @@ class ConnectRequestServiceTest {
assertThat(service.claim("nope", CLAIM_SECRET).outcome()).isEqualTo(ClaimOutcome.REJECTED);
}
// ---------------------------------------------------------------------------------------
private static ConnectRequest pending() {
ConnectRequest row = new ConnectRequest();
row.setRequestId("req");
@@ -64,6 +64,7 @@ class TeamBillingServiceMoreTest {
e.setTeamId(TEAM_ID);
e.setPaygSubscriptionId(subscriptionId);
e.setFreeUnitsRemaining(freeRemaining);
e.setFreeUnitsPeriodStart(TeamBillingService.calendarMonthWindow()[0]);
return e;
}
@@ -360,4 +361,132 @@ class TeamBillingServiceMoreTest {
assertThat(window[1]).isEqualTo(YearMonth.now().plusMonths(1).atDay(1).atStartOfDay());
}
}
@Nested
@DisplayName("compute: recurring free grant")
class RecurringFreeGrant {
private static final long GRANT = 500L;
private PaygTeamExtensions stamped(LocalDateTime stamp, long remaining) {
PaygTeamExtensions e = new PaygTeamExtensions();
e.setTeamId(TEAM_ID);
e.setFreeUnitsRemaining(remaining);
e.setFreeUnitsPeriodStart(stamp);
return e;
}
@Test
@DisplayName("a counter stamped with a past period reads as a fresh grant")
void staleStampReadsAsFreshGrant() {
stubGrant(GRANT);
// Nothing has persisted the reset yet, so the read has to show it anyway.
when(extensionsRepository.findById(TEAM_ID))
.thenReturn(
Optional.of(
stamped(
LocalDateTime.now().minusMonths(2).withDayOfMonth(1),
0L)));
TeamBillingContext ctx = service.forTeam(TEAM_ID);
assertThat(ctx.freeGrantUnits()).isEqualTo(GRANT);
assertThat(ctx.freeRemainingUnits()).isEqualTo(GRANT);
}
@Test
@DisplayName("a counter stamped with the current period reads as the stored balance")
void currentStampReadsStoredBalance() {
stubGrant(GRANT);
when(extensionsRepository.findById(TEAM_ID))
.thenReturn(
Optional.of(
stamped(TeamBillingService.calendarMonthWindow()[0], 120L)));
assertThat(service.forTeam(TEAM_ID).freeRemainingUnits()).isEqualTo(120L);
}
@Test
@DisplayName(
"an unstamped row — written before the grant recurred — reads as a fresh grant")
void nullStampReadsAsFreshGrant() {
stubGrant(GRANT);
when(extensionsRepository.findById(TEAM_ID)).thenReturn(Optional.of(stamped(null, 0L)));
assertThat(service.forTeam(TEAM_ID).freeRemainingUnits()).isEqualTo(GRANT);
}
@Test
@DisplayName("the grant resets on the Stripe window, not the calendar month")
void subscribedGrantFollowsTheStripeWindow() {
stubGrant(GRANT);
// Stamped for the calendar month, but this team's period is Stripe-anchored and starts
// mid-month, so the stamp belongs to the previous period and the grant resets.
LocalDateTime stripeStart = LocalDateTime.of(2026, 6, 10, 0, 0);
when(extensionsRepository.findById(TEAM_ID))
.thenReturn(Optional.of(subscribedRow(LocalDateTime.of(2026, 6, 1, 0, 0))));
when(subscriptionDao.findBilling("sub_1"))
.thenReturn(
Optional.of(
new SubscriptionBilling(
stripeStart,
stripeStart.plusMonths(1),
"price_1",
"active",
"usd",
new BigDecimal("2"))));
TeamBillingContext ctx = service.forTeam(TEAM_ID);
assertThat(ctx.periodStart()).isEqualTo(stripeStart);
assertThat(ctx.freeRemainingUnits()).isEqualTo(GRANT);
}
private PaygTeamExtensions subscribedRow(LocalDateTime stamp) {
PaygTeamExtensions e = stamped(stamp, 0L);
e.setPaygSubscriptionId("sub_1");
return e;
}
}
@Nested
@DisplayName("remainingForPeriod")
class RemainingForPeriodRule {
private final LocalDateTime period = LocalDateTime.of(2026, 8, 1, 0, 0);
@Test
@DisplayName("stale stamp yields the full grant; current stamp yields the stored balance")
void staleVersusCurrent() {
assertThat(
TeamBillingService.remainingForPeriod(
period.minusMonths(1), 0L, 500L, period))
.isEqualTo(500L);
assertThat(TeamBillingService.remainingForPeriod(period, 0L, 500L, period)).isZero();
assertThat(TeamBillingService.remainingForPeriod(period, 42L, 500L, period))
.isEqualTo(42L);
}
@Test
@DisplayName("a stamp in the future is never read as another grant")
void futureStampKeepsTheStoredBalance() {
assertThat(TeamBillingService.remainingForPeriod(period.plusDays(1), 7L, 500L, period))
.isEqualTo(7L);
}
@Test
@DisplayName("null stored balance and negative values floor at zero")
void nullAndNegativeBalances() {
assertThat(TeamBillingService.remainingForPeriod(period, null, 500L, period)).isZero();
assertThat(TeamBillingService.remainingForPeriod(period, -5L, 500L, period)).isZero();
assertThat(TeamBillingService.remainingForPeriod(null, 0L, -1L, period)).isZero();
}
@Test
@DisplayName("an unknown current period leaves the counter alone")
void nullCurrentPeriod() {
assertThat(TeamBillingService.isStale(null, null)).isFalse();
assertThat(TeamBillingService.remainingForPeriod(null, 3L, 500L, null)).isEqualTo(3L);
}
}
}
@@ -58,12 +58,14 @@ class TeamBillingServiceTest {
when(pricingPolicyService.getEffectivePolicy(TEAM_ID)).thenReturn(policy);
}
/** Stamped with the current period, so {@code freeRemaining} reads as the live balance. */
private PaygTeamExtensions ext(String subscriptionId, String customerId, long freeRemaining) {
PaygTeamExtensions ext = new PaygTeamExtensions();
ext.setTeamId(TEAM_ID);
ext.setPaygSubscriptionId(subscriptionId);
ext.setStripeCustomerId(customerId);
ext.setFreeUnitsRemaining(freeRemaining);
ext.setFreeUnitsPeriodStart(TeamBillingService.calendarMonthWindow()[0]);
return ext;
}
@@ -31,6 +31,8 @@ import org.springframework.transaction.support.TransactionSynchronization;
import org.springframework.transaction.support.TransactionSynchronizationManager;
import org.springframework.web.multipart.MultipartFile;
import stirling.software.saas.payg.billing.TeamBillingContext;
import stirling.software.saas.payg.billing.TeamBillingService;
import stirling.software.saas.payg.bundle.PrepaidBundleService;
import stirling.software.saas.payg.docs.DocumentClassifier;
import stirling.software.saas.payg.docs.DocumentMetrics;
@@ -72,8 +74,13 @@ class JobChargeServiceTest {
private PaygMeterReportingService meterReporter;
private WalletLedgerRepository ledgerRepo;
private PrepaidBundleService prepaidBundleService;
private TeamBillingService teamBillingService;
private JobChargeService service;
private static final LocalDateTime PERIOD_START = LocalDateTime.of(2026, 8, 1, 0, 0);
private static final long GRANT = 500L;
@BeforeEach
void setUp() {
jobService = Mockito.mock(JobService.class);
@@ -90,6 +97,8 @@ class JobChargeServiceTest {
// findByIdForUpdate defaults to Optional.empty() (Mockito) → no free grant consumed unless
// a test stubs the sidecar row. The free split is decided at openProcess time now, not at
// close, so the meter tests just set free_units_consumed on the shadow row directly.
teamBillingService = Mockito.mock(TeamBillingService.class);
when(teamBillingService.forTeam(Mockito.anyLong())).thenReturn(billingContext(GRANT));
service =
new JobChargeService(
jobService,
@@ -100,7 +109,22 @@ class JobChargeServiceTest {
teamExtRepo,
meterReporter,
ledgerRepo,
prepaidBundleService);
prepaidBundleService,
teamBillingService);
}
private static TeamBillingContext billingContext(long grant) {
return new TeamBillingContext(
false,
null,
PERIOD_START,
PERIOD_START.plusMonths(1),
grant,
grant,
null,
null,
null,
null);
}
@AfterEach
@@ -365,6 +389,7 @@ class JobChargeServiceTest {
PaygTeamExtensions ext = new PaygTeamExtensions();
ext.setTeamId(100L);
ext.setFreeUnitsRemaining(10L);
ext.setFreeUnitsPeriodStart(PERIOD_START);
when(teamExtRepo.findByIdForUpdate(100L)).thenReturn(Optional.of(ext));
service.openProcess(
@@ -381,6 +406,96 @@ class JobChargeServiceTest {
verify(teamExtRepo).save(ext);
}
@Test
void openProcess_firstChargeOfNewPeriod_resetsGrantAndRestamps(@TempDir Path tmp)
throws IOException {
// Grant exhausted last period, nothing run since. This charge persists the reset: counter
// back to the full grant, drawn from, and re-stamped so the next charge reads the balance.
PricingPolicy policy = stubPolicy(1, Map.of(JobSource.WEB, 10));
when(policyService.getEffectivePolicy(100L)).thenReturn(policy);
ProcessingJob newJob = openJob(UUID.randomUUID());
when(jobService.joinOrOpen(any(JobContext.class), anyList()))
.thenReturn(new JoinOrOpenResult(newJob, JoinOrOpenResult.Disposition.OPENED));
when(classifier.classify(any(MultipartFile.class), any(Path.class), eq(policy)))
.thenReturn(new DocumentMetrics(50, 1024L, "application/pdf", 4));
PaygTeamExtensions ext = new PaygTeamExtensions();
ext.setTeamId(100L);
ext.setFreeUnitsRemaining(0L);
ext.setFreeUnitsPeriodStart(PERIOD_START.minusMonths(1));
when(teamExtRepo.findByIdForUpdate(100L)).thenReturn(Optional.of(ext));
service.openProcess(
new ChargeContext(
42L, 100L, JobSource.WEB, ProcessType.SINGLE_TOOL, BillingCategory.API),
List.of(jobInput(tmp, "in.pdf", "application/pdf")));
ArgumentCaptor<PaygShadowCharge> captor = ArgumentCaptor.forClass(PaygShadowCharge.class);
verify(shadowRepo).save(captor.capture());
assertThat(captor.getValue().getFreeUnitsConsumed()).isEqualTo(4);
assertThat(ext.getFreeUnitsRemaining()).isEqualTo(GRANT - 4);
assertThat(ext.getFreeUnitsPeriodStart()).isEqualTo(PERIOD_START);
verify(teamExtRepo).save(ext);
}
@Test
void openProcess_unstampedRow_resetsToGrantAndStamps(@TempDir Path tmp) throws IOException {
// An unstamped row must read as owed a reset, not as an exhausted pool.
PricingPolicy policy = stubPolicy(1, Map.of(JobSource.WEB, 10));
when(policyService.getEffectivePolicy(100L)).thenReturn(policy);
ProcessingJob newJob = openJob(UUID.randomUUID());
when(jobService.joinOrOpen(any(JobContext.class), anyList()))
.thenReturn(new JoinOrOpenResult(newJob, JoinOrOpenResult.Disposition.OPENED));
when(classifier.classify(any(MultipartFile.class), any(Path.class), eq(policy)))
.thenReturn(new DocumentMetrics(50, 1024L, "application/pdf", 1));
PaygTeamExtensions ext = new PaygTeamExtensions();
ext.setTeamId(100L);
ext.setFreeUnitsRemaining(0L);
when(teamExtRepo.findByIdForUpdate(100L)).thenReturn(Optional.of(ext));
service.openProcess(
new ChargeContext(
42L, 100L, JobSource.WEB, ProcessType.SINGLE_TOOL, BillingCategory.API),
List.of(jobInput(tmp, "in.pdf", "application/pdf")));
ArgumentCaptor<PaygShadowCharge> captor = ArgumentCaptor.forClass(PaygShadowCharge.class);
verify(shadowRepo).save(captor.capture());
assertThat(captor.getValue().getFreeUnitsConsumed()).isEqualTo(1);
assertThat(ext.getFreeUnitsRemaining()).isEqualTo(GRANT - 1);
assertThat(ext.getFreeUnitsPeriodStart()).isEqualTo(PERIOD_START);
}
@Test
void openProcess_zeroGrantRollover_stampsWithoutDrawing(@TempDir Path tmp) throws IOException {
// A zero grant still advances the stamp, or every later charge re-evaluates a stale row.
when(teamBillingService.forTeam(100L)).thenReturn(billingContext(0L));
PricingPolicy policy = stubPolicy(1, Map.of(JobSource.WEB, 10));
when(policyService.getEffectivePolicy(100L)).thenReturn(policy);
ProcessingJob newJob = openJob(UUID.randomUUID());
when(jobService.joinOrOpen(any(JobContext.class), anyList()))
.thenReturn(new JoinOrOpenResult(newJob, JoinOrOpenResult.Disposition.OPENED));
when(classifier.classify(any(MultipartFile.class), any(Path.class), eq(policy)))
.thenReturn(new DocumentMetrics(50, 1024L, "application/pdf", 3));
PaygTeamExtensions ext = new PaygTeamExtensions();
ext.setTeamId(100L);
ext.setFreeUnitsRemaining(0L);
when(teamExtRepo.findByIdForUpdate(100L)).thenReturn(Optional.of(ext));
service.openProcess(
new ChargeContext(
42L, 100L, JobSource.WEB, ProcessType.SINGLE_TOOL, BillingCategory.API),
List.of(jobInput(tmp, "in.pdf", "application/pdf")));
ArgumentCaptor<PaygShadowCharge> captor = ArgumentCaptor.forClass(PaygShadowCharge.class);
verify(shadowRepo).save(captor.capture());
assertThat(captor.getValue().getFreeUnitsConsumed()).isZero();
assertThat(ext.getFreeUnitsRemaining()).isZero();
assertThat(ext.getFreeUnitsPeriodStart()).isEqualTo(PERIOD_START);
verify(teamExtRepo).save(ext);
}
@Test
void openProcess_grantStraddle_drawsRemainderFreeAndBillsTheRest(@TempDir Path tmp)
throws IOException {
@@ -396,6 +511,7 @@ class JobChargeServiceTest {
PaygTeamExtensions ext = new PaygTeamExtensions();
ext.setTeamId(100L);
ext.setFreeUnitsRemaining(3L);
ext.setFreeUnitsPeriodStart(PERIOD_START);
when(teamExtRepo.findByIdForUpdate(100L)).thenReturn(Optional.of(ext));
service.openProcess(
@@ -426,6 +542,7 @@ class JobChargeServiceTest {
PaygTeamExtensions ext = new PaygTeamExtensions();
ext.setTeamId(100L);
ext.setFreeUnitsRemaining(0L);
ext.setFreeUnitsPeriodStart(PERIOD_START);
when(teamExtRepo.findByIdForUpdate(100L)).thenReturn(Optional.of(ext));
service.openProcess(
@@ -544,8 +661,8 @@ class JobChargeServiceTest {
assertThat(refund.getReferenceId()).isEqualTo(jobId.toString());
assertThat(refund.getPolicyId()).isEqualTo(7L);
assertThat(refund.getBillingCategory()).isEqualTo(BillingCategory.API);
// This row consumed no free units, so the grant counter is left alone.
verify(teamExtRepo, never()).restoreFreeUnits(eq(100L), Mockito.anyLong());
// No free units consumed, so the grant counter is never loaded.
verify(teamExtRepo, never()).findByIdForUpdate(100L);
}
@Test
@@ -556,10 +673,35 @@ class JobChargeServiceTest {
PaygShadowCharge row = chargedShadowRow(jobId, 100L, 10, 3, BillingCategory.API);
when(shadowRepo.findFirstByJobIdOrderByIdAsc(jobId)).thenReturn(Optional.of(row));
when(jobRepo.findById(jobId)).thenReturn(Optional.of(openJob(jobId)));
PaygTeamExtensions ext = new PaygTeamExtensions();
ext.setTeamId(100L);
ext.setFreeUnitsRemaining(GRANT - 3);
ext.setFreeUnitsPeriodStart(PERIOD_START);
when(teamExtRepo.findByIdForUpdate(100L)).thenReturn(Optional.of(ext));
service.markFirstStepFailed(jobId, "first-step-5xx:503");
verify(teamExtRepo).restoreFreeUnits(100L, 3L);
assertThat(ext.getFreeUnitsRemaining()).isEqualTo(GRANT);
verify(teamExtRepo).save(ext);
}
@Test
void markFirstStepFailed_refundAfterPeriodTurned_doesNotExceedTheGrant() {
// The charge's period is over and the grant already reset, so the restore is capped.
UUID jobId = UUID.randomUUID();
PaygShadowCharge row = chargedShadowRow(jobId, 100L, 10, 3, BillingCategory.API);
when(shadowRepo.findFirstByJobIdOrderByIdAsc(jobId)).thenReturn(Optional.of(row));
when(jobRepo.findById(jobId)).thenReturn(Optional.of(openJob(jobId)));
PaygTeamExtensions ext = new PaygTeamExtensions();
ext.setTeamId(100L);
ext.setFreeUnitsRemaining(0L);
ext.setFreeUnitsPeriodStart(PERIOD_START.minusMonths(1));
when(teamExtRepo.findByIdForUpdate(100L)).thenReturn(Optional.of(ext));
service.markFirstStepFailed(jobId, "first-step-5xx:503");
assertThat(ext.getFreeUnitsRemaining()).isEqualTo(GRANT);
assertThat(ext.getFreeUnitsPeriodStart()).isEqualTo(PERIOD_START);
}
@Test
@@ -886,6 +1028,7 @@ class JobChargeServiceTest {
ext.setStripeCustomerId("cus_x");
ext.setPaygSubscriptionId("sub_x");
ext.setFreeUnitsRemaining(0L);
ext.setFreeUnitsPeriodStart(PERIOD_START);
when(teamExtRepo.findByIdForUpdate(teamId)).thenReturn(Optional.of(ext));
when(teamExtRepo.findById(teamId)).thenReturn(Optional.of(ext));
when(shadowRepo.findFirstByJobIdOrderByIdAsc(jobId))
@@ -930,6 +1073,7 @@ class JobChargeServiceTest {
PaygTeamExtensions ext = new PaygTeamExtensions();
ext.setTeamId(teamId);
ext.setFreeUnitsRemaining(50L);
ext.setFreeUnitsPeriodStart(PERIOD_START);
when(teamExtRepo.findByIdForUpdate(teamId)).thenReturn(Optional.of(ext));
when(teamExtRepo.findById(teamId)).thenReturn(Optional.of(ext));
when(shadowRepo.findFirstByJobIdOrderByIdAsc(jobId))
@@ -974,6 +1118,7 @@ class JobChargeServiceTest {
ext.setStripeCustomerId("cus_x");
ext.setPaygSubscriptionId("sub_x");
ext.setFreeUnitsRemaining(0L);
ext.setFreeUnitsPeriodStart(PERIOD_START);
when(teamExtRepo.findByIdForUpdate(teamId)).thenReturn(Optional.of(ext));
when(teamExtRepo.findById(teamId)).thenReturn(Optional.of(ext));
when(shadowRepo.findFirstByJobIdOrderByIdAsc(jobId))
@@ -1032,8 +1177,6 @@ class JobChargeServiceTest {
return row;
}
// --- helpers --------------------------------------------------------------------------------
private static PricingPolicy stubPolicy(int minCharge, Map<JobSource, Integer> stepLimits) {
PricingPolicy p = new PricingPolicy();
p.setId(42L);
@@ -26,8 +26,8 @@ import stirling.software.saas.payg.repository.WalletPolicyRepository;
import stirling.software.saas.payg.wallet.WalletPolicy;
/**
* Unit tests for {@link EntitlementService}. Two branches (design 2026-06-11 — the free allowance
* is a one-time lifetime grant):
* Unit tests for {@link EntitlementService}. Two branches (the free allowance is a per-period
* grant, projected onto the current period by {@code TeamBillingService} before it gets here):
*
* <ul>
* <li><b>Unsubscribed</b> — gated by the grant. Cap = grant size, spend = {@code grant
@@ -96,7 +96,6 @@ class EntitlementServiceTest {
assertThat(snap.periodCapUnits()).isEqualTo(2000L);
assertThat(snap.periodSpendUnits()).isEqualTo(500L);
// 500/2000 = 25% — FULL
assertThat(snap.state()).isEqualTo(EntitlementState.FULL);
}
+5 -1
View File
@@ -452,8 +452,12 @@ subprojects {
if (name == 'stirling-pdf') {
apply plugin: 'org.springdoc.openapi-gradle-plugin'
def openApiPort = rootProject.findProperty('openApiPort')?.toString()
?: new ServerSocket(0).withCloseable { it.localPort }.toString()
openApi {
apiDocsUrl = "http://localhost:8080/v1/api-docs"
apiDocsUrl = "http://localhost:${openApiPort}/v1/api-docs"
customBootRun { args = ["--server.port=${openApiPort}"] }
outputDir = file("$projectDir")
outputFileName = "SwaggerDoc.json"
waitTimeInSeconds = 60 // Increase the wait time to 60 seconds
+14
View File
@@ -48,9 +48,23 @@ ENV STIRLING_FLAVOR=${STIRLING_FLAVOR}
# portal or AI layers change; defaults false so normal builds skip the extra app.
ARG BUILD_PORTAL=false
# Which Stirling account the portal connects to. Build-time because Vite inlines VITE_* into the
# bundle; there is no runtime override. Empty leaves the committed .env.proprietary defaults, which
# is what an ordinary image wants: no Stirling account and no connect flow. The publishable key is
# client-side by design, not a secret. Pass the URL and the key from the same Supabase project or
# the browser accepts the pair and Supabase rejects it, which surfaces later as "session expired".
ARG VITE_SUPABASE_URL=""
ARG VITE_SUPABASE_PUBLISHABLE_DEFAULT_KEY=""
ARG VITE_SAAS_API_URL=""
# Bundle only the JPDFium native for this image's target arch.
ARG TARGETARCH
# Exported only when non-empty: Vite reads process.env ahead of the .env files, so exporting an
# empty value would blank the committed default rather than fall back to it.
RUN JPDFIUM_PLATFORM="$([ "$TARGETARCH" = arm64 ] && echo linux-arm64 || echo linux-x64)" && \
if [ -n "${VITE_SUPABASE_URL}" ]; then export VITE_SUPABASE_URL="${VITE_SUPABASE_URL}"; fi; \
if [ -n "${VITE_SUPABASE_PUBLISHABLE_DEFAULT_KEY}" ]; then export VITE_SUPABASE_PUBLISHABLE_DEFAULT_KEY="${VITE_SUPABASE_PUBLISHABLE_DEFAULT_KEY}"; fi; \
if [ -n "${VITE_SAAS_API_URL}" ]; then export VITE_SAAS_API_URL="${VITE_SAAS_API_URL}"; fi; \
STIRLING_FLAVOR=${STIRLING_FLAVOR} \
gradle clean build \
-PbuildWithFrontend=true \
+4 -1
View File
@@ -30,7 +30,7 @@ from stirling.contracts import (
)
from stirling.contracts.pdf_create import PdfCreateOrchestrateResponse
from stirling.models import ApiModel
from stirling.services import AppRuntime
from stirling.services import AppRuntime, language_directive, set_reply_locale
logger = logging.getLogger(__name__)
@@ -153,6 +153,8 @@ class OrchestratorAgent:
)
async def handle(self, request: OrchestratorRequest) -> OrchestratorResponse:
# Bound once; delegates and worker tasks inherit it.
set_reply_locale(request.locale)
logger.info(
"[orchestrator] handle: files=%s resume_with=%s artifacts=%s msg=%r",
[file.name for file in request.files],
@@ -270,6 +272,7 @@ class OrchestratorAgent:
f"User message: {request.user_message}\n"
f"Files: {format_file_names(request.files)}\n"
f"Available artifacts:\n{artifact_summary}"
f"\n{language_directive()}"
)
def _describe_artifacts(self, request: OrchestratorRequest) -> str:
@@ -30,7 +30,7 @@ from stirling.contracts.pdf_comments import (
)
from stirling.logging import Pretty
from stirling.models import ApiModel
from stirling.services import AppRuntime
from stirling.services import AppRuntime, language_directive
logger = logging.getLogger(__name__)
@@ -160,6 +160,8 @@ class PdfCommentAgent:
]
for index, chunk in enumerate(request.chunks):
lines.append(f"[{index}] page={chunk.page + 1} text={json.dumps(chunk.text)}")
# Last, after the untrusted chunk text.
lines.append(f"\n{language_directive()}")
return "\n".join(lines)
@staticmethod
@@ -45,7 +45,7 @@ from stirling.contracts.pdf_create import (
WrittenSections,
)
from stirling.models.agent_tool_models import AgentToolId, CreatePdfFromHtmlAgentParams
from stirling.services import AppRuntime
from stirling.services import AppRuntime, language_directive
logger = logging.getLogger(__name__)
@@ -260,6 +260,7 @@ def _build_sections_prompt(meta: DocumentMeta, user_request: str, history: str)
lines.append(f"\nConversation history:\n{history}")
lines.append(f"\nUser request: {user_request}")
lines.append(f"\n{language_directive()}")
return "\n".join(lines)
@@ -292,6 +293,7 @@ def _build_writer_prompt(plan: DocumentPlan, chunk: _Chunk) -> str:
for point in s.key_points:
lines.append(f" - {point}")
lines.append(f"\n{language_directive()}")
return "\n".join(lines)
@@ -338,6 +340,7 @@ class PdfCreateAgent:
# ── Phase 1: plan meta ─────────────────────────────────────────────────
logger.info("[pdf-create] phase 1/6: planning document meta")
meta_prompt = f"Conversation history:\n{history}\n\nUser request: {request.user_message}"
meta_prompt += f"\n\n{language_directive()}"
meta_result = await self._meta_planner.run(meta_prompt)
meta = meta_result.output
+11 -29
View File
@@ -27,7 +27,7 @@ from stirling.contracts import (
)
from stirling.logging import Pretty
from stirling.models import OPERATIONS, ApiModel, ParamToolModel, ToolEndpoint
from stirling.services import AppRuntime, ToolChainStep, blocking, validate_tool_chain
from stirling.services import AppRuntime, ToolChainStep, blocking, language_directive, validate_tool_chain
logger = logging.getLogger(__name__)
@@ -282,9 +282,7 @@ class PdfEditAgent:
unavailable_operations,
allow_need_content=can_request_content,
)
return await agent.select(
self._build_selection_prompt(request, supported_operations, unavailable_operations, repair_note)
)
return await agent.select(self._build_selection_prompt(request, supported_operations, repair_note))
def _build_selection_agent(
self,
@@ -330,7 +328,6 @@ class PdfEditAgent:
self,
request: PdfEditRequest,
supported_operations: Iterable[ToolEndpoint],
unavailable_operations: Iterable[ToolEndpoint],
repair_note: str = "",
) -> str:
repair_line = (
@@ -343,35 +340,20 @@ class PdfEditAgent:
if repair_note
else ""
)
unavailable_line = (
"Unavailable operations (exist but not currently usable): "
f"{self._get_operations_prompt(unavailable_operations)}\n"
if unavailable_operations
else ""
)
return (
f"Conversation history:\n{format_conversation_history(request.conversation_history)}\n"
f"User request: {request.user_message}\n"
f"Files: {format_file_names(request.files)}\n"
f"Supported operations:\n{self._get_supported_operations_prompt(supported_operations)}\n"
f"{unavailable_line}"
f"{repair_line}"
f"Extracted page text:\n{format_page_text(request.page_text)}"
f"Conversation history:\n{format_conversation_history(request.conversation_history)}\n"
f"Files: {format_file_names(request.files)}\n"
f"Extracted page text:\n{format_page_text(request.page_text)}\n"
f"{language_directive()}\n"
f"User request: {request.user_message}"
)
# Endpoints that exist on the server and are callable via the direct API or the manual UI,
# but are never offered to the AI agent as a routing option.
#
# Why: REDACT_EXECUTE is the preferred AI-driven redaction route. AUTO_REDACT and REDACT are
# legacy endpoints that remain fully functional for human callers (the manual redact UI, direct
# API consumers, pipelines) but would produce a worse experience if the AI routed to them —
# they accept a simpler, less expressive schema and pre-date the unified operation model.
# Hiding them here channels all AI redaction traffic through REDACT_EXECUTE without disabling
# the legacy endpoints for anyone else.
#
# How to reuse: add an endpoint here whenever a legacy endpoint has a preferred replacement
# that the AI should use exclusively. The endpoint remains live on the server; only the AI
# planner is prevented from selecting it.
# Hidden from the AI planner only; still live for the manual UI, direct API and pipelines.
# AUTO_REDACT and REDACT pre-date the unified operation model and take a less expressive
# schema, so AI redaction is channelled through REDACT_EXECUTE. Add an endpoint here when a
# legacy one has a preferred replacement the AI should use exclusively.
_AGENT_HIDDEN_ENDPOINTS: frozenset[ToolEndpoint] = frozenset({ToolEndpoint.AUTO_REDACT, ToolEndpoint.REDACT})
def _classify_operations(self, request: PdfEditRequest) -> tuple[list[ToolEndpoint], list[ToolEndpoint]]:
+3 -1
View File
@@ -29,7 +29,7 @@ from stirling.contracts import (
from stirling.documents import RagCapability
from stirling.models import PrincipalId
from stirling.models.agent_tool_models import AgentToolId, MathAuditorAgentParams
from stirling.services import AppRuntime, require_current_user_id
from stirling.services import AppRuntime, language_directive, require_current_user_id
logger = logging.getLogger(__name__)
@@ -223,6 +223,7 @@ class PdfQuestionAgent:
forbids invented figures; the LLM only restates Verdict facts.
"""
prompt = f"User question:\n{user_message}\n\nMath audit Verdict (JSON):\n{verdict.model_dump_json()}"
prompt += f"\n\n{language_directive()}"
result = await self._math_synth_agent.run(prompt)
return result.output
@@ -233,4 +234,5 @@ class PdfQuestionAgent:
f"Files: {format_file_names(request.files)}\n"
f"Question: {request.question}\n"
"Pick the right retrieval tool for this question, then answer from what it returns."
f"\n{language_directive()}"
)
+3 -1
View File
@@ -56,7 +56,7 @@ from stirling.models.agent_tool_models import (
PdfCommentAgentParams,
)
from stirling.models.tool_models import AddCommentsParams
from stirling.services import AppRuntime, require_current_user_id
from stirling.services import AppRuntime, language_directive, require_current_user_id
# Fallback right-margin placement used when a finding has no usable
# anchor text. A4/Letter portrait assumed.
@@ -209,6 +209,7 @@ class PdfReviewAgent:
placement geometry to produce the JSON the ``add-comments`` tool wants.
"""
prompt = f"User review request:\n{user_message}\n\nMath audit Verdict (JSON):\n{verdict.model_dump_json()}"
prompt += f"\n\n{language_directive()}"
result = await self._localiser_agent.run(prompt)
specs = self._build_comment_specs(verdict, result.output.comments)
serialised = [spec.model_dump(by_alias=True, exclude_none=True) for spec in specs]
@@ -238,6 +239,7 @@ class PdfReviewAgent:
prompt = (
f"<user_message>{_escape_for_tag(user_message)}</user_message>\n"
f"<verdict>{_escape_for_tag(report.model_dump_json())}</verdict>"
f"\n{language_directive()}"
)
result = await self._contradiction_localiser.run(prompt)
specs = self._build_paired_comment_specs(report, result.output.comments)
+3 -1
View File
@@ -21,7 +21,7 @@ from stirling.contracts import (
format_conversation_history,
)
from stirling.models import ApiModel
from stirling.services import AppRuntime
from stirling.services import AppRuntime, language_directive
class UserSpecMetadata(ApiModel):
@@ -98,6 +98,7 @@ class UserSpecAgent:
f"Edit plan summary:\n{edit_plan.summary}\n\n"
f"Edit plan rationale:\n{edit_plan.rationale or 'None'}\n\n"
f"Edit plan steps:\n{edit_plan.model_dump_json(indent=2)}"
f"\n\n{language_directive()}"
)
def _build_revision_prompt(self, request: AgentRevisionRequest, edit_plan: EditPlanResponse) -> str:
@@ -108,6 +109,7 @@ class UserSpecAgent:
f"Edit plan summary:\n{edit_plan.summary}\n\n"
f"Edit plan rationale:\n{edit_plan.rationale or 'None'}\n\n"
f"Edit plan steps:\n{edit_plan.model_dump_json(indent=2)}"
f"\n\n{language_directive()}"
)
async def _build_edit_plan(
@@ -42,6 +42,8 @@ class OrchestratorRequest(ApiModel):
conversation_history: list[ConversationMessage] = Field(default_factory=list)
artifacts: list[WorkflowArtifact] = Field(default_factory=list)
resume_with: SupportedCapability | None = None
# Reply language (IETF tag); unset falls back to the message's own language.
locale: str | None = None
# See `PdfEditRequest.enabled_endpoints`.
enabled_endpoints: Annotated[list[ToolEndpoint], BeforeValidator(drop_unknown_tool_endpoints)] = Field(
default_factory=list
+14
View File
@@ -491,6 +491,15 @@ class EmlToPdfParams(ApiModel):
)
class EncodeCharcodesParams(ApiModel):
font_name: str | None = None
font_sha256: str | None = None
locator_char: str | None = None
page_index: int | None = None
pdf_base64: str | None = None
text: str | None = None
class ExtractAttachmentsParams(ApiModel):
pass
@@ -1547,6 +1556,7 @@ class Model(
| EditTextParams
| MergePdfsParams
| MultiPageLayoutParams
| EncodeCharcodesParams
| PdfToSinglePageParams
| RearrangePagesParams
| RemoveImagePdfParams
@@ -1623,6 +1633,7 @@ class Model(
| EditTextParams
| MergePdfsParams
| MultiPageLayoutParams
| EncodeCharcodesParams
| PdfToSinglePageParams
| RearrangePagesParams
| RemoveImagePdfParams
@@ -1700,6 +1711,7 @@ type ParamToolModel = (
| EditTextParams
| MergePdfsParams
| MultiPageLayoutParams
| EncodeCharcodesParams
| PdfToSinglePageParams
| RearrangePagesParams
| RemoveImagePdfParams
@@ -1778,6 +1790,7 @@ class ToolEndpoint(StrEnum):
EDIT_TEXT = "/api/v1/general/edit-text"
MERGE_PDFS = "/api/v1/general/merge-pdfs"
MULTI_PAGE_LAYOUT = "/api/v1/general/multi-page-layout"
ENCODE_CHARCODES = "/api/v1/general/pdf-text-editor/encode-charcodes"
PDF_TO_SINGLE_PAGE = "/api/v1/general/pdf-to-single-page"
REARRANGE_PAGES = "/api/v1/general/rearrange-pages"
REMOVE_IMAGE_PDF = "/api/v1/general/remove-image-pdf"
@@ -1854,6 +1867,7 @@ OPERATIONS: dict[ToolEndpoint, ParamToolModelType] = {
ToolEndpoint.EDIT_TEXT: EditTextParams,
ToolEndpoint.MERGE_PDFS: MergePdfsParams,
ToolEndpoint.MULTI_PAGE_LAYOUT: MultiPageLayoutParams,
ToolEndpoint.ENCODE_CHARCODES: EncodeCharcodesParams,
ToolEndpoint.PDF_TO_SINGLE_PAGE: PdfToSinglePageParams,
ToolEndpoint.REARRANGE_PAGES: RearrangePagesParams,
ToolEndpoint.REMOVE_IMAGE_PDF: RemoveImagePdfParams,
+3
View File
@@ -1,5 +1,6 @@
"""Shared services used by the Stirling AI runtime."""
from .language import language_directive, set_reply_locale
from .progress import (
ProgressEmitter,
emit_progress,
@@ -20,9 +21,11 @@ __all__ = [
"build_runtime",
"current_user_id",
"emit_progress",
"language_directive",
"require_current_user_id",
"reset_progress_emitter",
"set_progress_emitter",
"set_reply_locale",
"setup_posthog_tracking",
"validate_tool_chain",
]
+23
View File
@@ -0,0 +1,23 @@
"""Per-request reply language, bound by the orchestrator, read by prompt builders."""
from __future__ import annotations
from contextvars import ContextVar
_locale: ContextVar[str | None] = ContextVar("stirling_reply_locale", default=None)
def set_reply_locale(locale: str | None) -> None:
_locale.set(locale)
def language_directive() -> str:
"""Prompt line pinning the reply language; append to any user-facing prompt."""
locale = _locale.get()
if not locale:
return "Write anything the user will read in the same language as their message."
return (
f"Write anything the user will read in the language of locale '{locale}', whatever "
"language this prompt, the documents, or the tool output are in. Only a different "
"language the user explicitly asks for overrides this."
)
+7 -6
View File
@@ -144,7 +144,7 @@ def test_selection_prompt_says_nothing_about_output_formats(runtime: AppRuntime)
# Compatibility is only raised once a plan has actually failed, so the operation list stays
# about what each tool does. Leaking format hints here re-inflates an already large prompt.
agent = StubPdfEditAgent(runtime, _ANY_SELECTION)
prompt = agent._build_selection_prompt(PdfEditRequest(user_message="anything", files=[]), list(OPERATIONS), [])
prompt = agent._build_selection_prompt(PdfEditRequest(user_message="anything", files=[]), list(OPERATIONS))
assert "outputs:" not in prompt
assert "IMAGE (several files)" not in prompt
@@ -156,7 +156,6 @@ def test_repair_prompt_offers_reorder_or_telling_the_user(runtime: AppRuntime) -
prompt = agent._build_selection_prompt(
PdfEditRequest(user_message="anything", files=[]),
list(OPERATIONS),
[],
"step 3 (SANITIZE_PDF) accepts PDF but the previous step produces IMAGE.",
)
assert "SANITIZE_PDF" in prompt
@@ -494,11 +493,13 @@ def test_pdf_edit_selection_prompt_includes_unavailable_operations(runtime: AppR
)
supported, unavailable = agent._classify_operations(request)
prompt = agent._build_selection_prompt(request, supported, unavailable)
selection_agent = agent._build_selection_agent(supported, unavailable, allow_need_content=False)
system_prompt = "".join(selection_agent.agent._system_prompts)
prompt = agent._build_selection_prompt(request, supported)
assert "Unavailable operations" in prompt
assert "OCR_PDF" in prompt
assert ToolEndpoint.OCR_PDF.value in prompt
assert "NOT currently available" in system_prompt
assert "OCR_PDF" in system_prompt
assert "OCR_PDF" not in prompt
@pytest.mark.anyio
+66
View File
@@ -0,0 +1,66 @@
from __future__ import annotations
from collections.abc import Iterator
from typing import Any, cast
import pytest
from stirling.agents import OrchestratorAgent
from stirling.agents.pdf_questions import PdfQuestionAgent
from stirling.contracts import (
OrchestratorRequest,
PdfQuestionAnswerResponse,
PdfQuestionRequest,
SupportedCapability,
)
from stirling.services import language_directive, set_reply_locale
from stirling.services.runtime import AppRuntime
@pytest.fixture(autouse=True)
def reset_locale() -> Iterator[None]:
set_reply_locale(None)
yield
set_reply_locale(None)
def test_directive_falls_back_to_the_message_language() -> None:
assert "same language as their message" in language_directive()
def test_directive_pins_the_bound_locale() -> None:
set_reply_locale("fr-FR")
assert "'fr-FR'" in language_directive()
def test_orchestrator_request_carries_the_locale() -> None:
assert OrchestratorRequest.model_validate({"userMessage": "hi", "locale": "de-DE"}).locale == "de-DE"
assert OrchestratorRequest.model_validate({"userMessage": "hi"}).locale is None
def test_question_prompt_carries_the_directive() -> None:
set_reply_locale("es-ES")
# _build_prompt ignores self, so call it off the class.
prompt = PdfQuestionAgent._build_prompt(cast(Any, None), PdfQuestionRequest(question="¿Cuántas páginas?"))
assert "'es-ES'" in prompt
@pytest.mark.anyio
async def test_handle_binds_the_locale_for_delegates(runtime: AppRuntime, monkeypatch: pytest.MonkeyPatch) -> None:
"""The resume path reaches a delegate with the request's locale already bound."""
agent = OrchestratorAgent(runtime)
seen: list[str] = []
async def capture(request: OrchestratorRequest) -> PdfQuestionAnswerResponse:
seen.append(language_directive())
return PdfQuestionAnswerResponse(answer="ok")
monkeypatch.setattr(agent, "_run_pdf_question", capture)
await agent.handle(
OrchestratorRequest(
user_message="Combien de pages ?",
locale="fr-FR",
resume_with=SupportedCapability.PDF_QUESTION,
)
)
assert "'fr-FR'" in seen[0]
+16 -5
View File
@@ -25,6 +25,10 @@ const chromiumViewport = {
viewport: STUBBED_VIEWPORT,
};
// Dedicated dev-server port via V2_PORT so local runs don't collide with a
// vite already on 5173 from other parallel work. Defaults to 5173.
const DEV_PORT = process.env.V2_PORT ?? "5173";
export default defineConfig({
testDir: "./src/core/tests",
testMatch: "**/*.spec.ts",
@@ -49,7 +53,7 @@ export default defineConfig({
expect: { timeout: 10_000 },
use: {
baseURL: process.env.PLAYWRIGHT_BASE_URL ?? "http://localhost:5173",
baseURL: process.env.PLAYWRIGHT_BASE_URL ?? `http://localhost:${DEV_PORT}`,
trace: "on-first-retry",
screenshot: "only-on-failure",
video: "on-first-retry",
@@ -107,7 +111,14 @@ export default defineConfig({
{
name: "stubbed-webkit",
testDir: "./src/core/tests/stubbed",
use: { ...devices["Desktop Safari"], viewport: STUBBED_VIEWPORT },
// Desktop Safari ships deviceScaleFactor 2; the editor now renders
// bitmaps at dpr x zoom, so leaving it would 4x every page raster in
// this suite. The HiDPI spec opts into 2x deliberately where it matters.
use: {
...devices["Desktop Safari"],
viewport: STUBBED_VIEWPORT,
deviceScaleFactor: 1,
},
},
],
@@ -117,9 +128,9 @@ export default defineConfig({
// blew the 30s navigationTimeout under --workers=3 - see
// all-tool-pages-load.spec.ts). Locally, keep `vite` dev for HMR.
command: process.env.CI
? "npx vite preview --port 5173 --strictPort"
: "npx vite",
url: "http://localhost:5173",
? `npx vite preview --port ${DEV_PORT} --strictPort`
: `npx vite --port ${DEV_PORT} --strictPort`,
url: `http://localhost:${DEV_PORT}`,
reuseExistingServer: !process.env.CI,
timeout: 120_000,
},
@@ -0,0 +1,94 @@
Copyright 2014-2021 Adobe (http://www.adobe.com/), with Reserved Font Name 'Noto Sans'.
Copyright 2014-2021 Google Inc (http://www.google.com/), with Reserved Font Name 'Noto Sans'.
This Font Software is licensed under the SIL Open Font License, Version 1.1.
This license is copied below, and is also available with a FAQ at:
http://scripts.sil.org/OFL
-----------------------------------------------------------
SIL OPEN FONT LICENSE Version 1.1 - 26 February 2007
-----------------------------------------------------------
PREAMBLE
The goals of the Open Font License (OFL) are to stimulate worldwide
development of collaborative font projects, to support the font creation
efforts of academic and linguistic communities, and to provide a free and
open framework in which fonts may be shared and improved in partnership
with others.
The OFL allows the licensed fonts to be used, studied, modified and
redistributed freely as long as they are not sold by themselves. The
fonts, including any derivative works, can be bundled, embedded,
redistributed and/or sold with any software provided that any reserved
names are not used by derivative works. The fonts and derivatives,
however, cannot be released under any other type of license. The
requirement for fonts to remain under this license does not apply
to any document created using the fonts or their derivatives.
DEFINITIONS
"Font Software" refers to the set of files released by the Copyright
Holder(s) under this license and clearly marked as such. This may
include source files, build scripts and documentation.
"Reserved Font Name" refers to any names specified as such after the
copyright statement(s).
"Original Version" refers to the collection of Font Software components as
distributed by the Copyright Holder(s).
"Modified Version" refers to any derivative made by adding to, deleting,
or substituting -- in part or in whole -- any of the components of the
Original Version, by changing formats or by porting the Font Software to a
new environment.
"Author" refers to any designer, engineer, programmer, technical
writer or other person who contributed to the Font Software.
PERMISSION & CONDITIONS
Permission is hereby granted, free of charge, to any person obtaining
a copy of the Font Software, to use, study, copy, merge, embed, modify,
redistribute, and sell modified and unmodified copies of the Font
Software, subject to the following conditions:
1) Neither the Font Software nor any of its individual components,
in Original or Modified Versions, may be sold by itself.
2) Original or Modified Versions of the Font Software may be bundled,
redistributed and/or sold with any software, provided that each copy
contains the above copyright notice and this license. These can be
included either as stand-alone text files, human-readable headers or
in the appropriate machine-readable metadata fields within text or
binary files as long as those fields can be easily viewed by the user.
3) No Modified Version of the Font Software may use the Reserved Font
Name(s) unless explicit written permission is granted by the corresponding
Copyright Holder. This restriction only applies to the primary font name as
presented to the users.
4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font
Software shall not be used to promote, endorse or advertise any
Modified Version, except to acknowledge the contribution(s) of the
Copyright Holder(s) and the Author(s) or with their explicit written
permission.
5) The Font Software, modified or unmodified, in part or in whole,
must be distributed entirely under this license, and must not be
distributed under any other license. The requirement for fonts to
remain under this license does not apply to any document created
using the Font Software.
TERMINATION
This license becomes null and void if any of the above conditions are
not met.
DISCLAIMER
THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT
OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE
COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL
DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM
OTHER DEALINGS IN THE FONT SOFTWARE.
Binary file not shown.
File diff suppressed because it is too large Load Diff
@@ -6,6 +6,12 @@
"permissions": [
"core:default",
"core:window:allow-destroy",
"core:window:allow-minimize",
"core:window:allow-toggle-maximize",
"core:window:allow-internal-toggle-maximize",
"core:window:allow-close",
"core:window:allow-is-maximized",
"core:window:allow-start-dragging",
"http:default",
{
"identifier": "http:allow-fetch",
@@ -39,6 +45,18 @@
"identifier": "fs:allow-read-file",
"allow": [{ "path": "**" }]
},
{
"identifier": "fs:allow-read-dir",
"allow": [{ "path": "**" }]
},
{
"identifier": "fs:allow-stat",
"allow": [{ "path": "**" }]
},
{
"identifier": "fs:allow-mkdir",
"allow": [{ "path": "**" }]
},
{
"identifier": "fs:allow-write-file",
"allow": [{ "path": "**" }]
@@ -48,8 +48,11 @@ fn build_window(app: &AppHandle, label: &str, url: &str) -> Result<WebviewWindow
// dir (and thus IndexedDB / localStorage / cookies). macOS (WKWebView) and
// Linux (WebKitGTK) don't have this constraint, so the arg is Windows-only.
#[cfg(target_os = "windows")]
let builder =
builder.additional_browser_args("--enable-features=CertVerifierBuiltinFeature");
let builder = builder
.additional_browser_args("--enable-features=CertVerifierBuiltinFeature")
// Windows: no native title bar; the frontend draws its own (WindowTitleBar).
// macOS/Linux keep native decorations.
.decorations(false);
builder.build().map_err(|e| e.to_string())
}
+11
View File
@@ -146,6 +146,17 @@ pub fn run() {
.setup(|app| {
add_log("🚀 Tauri app setup started".to_string());
// Windows: drop the native title bar so the in-app custom title bar
// (window controls + drag region) takes over. Runtime toggle because the
// main window is defined in tauri.conf.json; spawned windows set it at
// build time in window.rs. macOS/Linux keep their native decorations.
#[cfg(target_os = "windows")]
{
if let Some(window) = app.get_webview_window(MAIN_WINDOW_LABEL) {
let _ = window.set_decorations(false);
}
}
// Files passed on the command line at first launch load into the main
// window once the frontend mounts.
let args: Vec<String> = std::env::args().collect();
@@ -5,7 +5,7 @@ import {
} from "@app/components/onboarding/onboardingSlideTypes";
export interface SaasFlowInputs {
/** Free-tier wallet with one-time allowance remaining — show the usage meter. */
/** Free-tier wallet with allowance remaining this period — show the usage meter. */
showUsageSlide: boolean;
/** Team leaders only — invited members and anonymous guests skip the team slide. */
showTeamSlide: boolean;
@@ -138,7 +138,7 @@ export function FreeLimitReachedModal({ onClose }: FreeLimitReachedModalProps) {
<div className={`${styles.bodyCopy} ${styles.bodyCopyInner}`}>
{t(
"plan.freeLimit.message",
"That's your whole free allowance for automation, AI and the API. Seriously impressive! Keep the momentum going for just pennies a day.",
"That's your whole free allowance for automation, AI and the API this month. Seriously impressive! It resets next month, or keep the momentum going now for just pennies a day.",
)}
</div>
</div>
@@ -9,14 +9,13 @@
* watermarks, compression — are unmetered, no matter where they're triggered
* from. The distinction is the <em>type of work</em> (manual tool vs
* automation / AI / API), not where the click happens, because automation and
* AI also have UI surfaces. The one-time free grant (default 500) applies
* <em>only</em> to the three billable categories — it is a lifetime allowance,
* not a monthly one, and a team keeps any unused portion after subscribing.
* AI also have UI surfaces. The free grant applies <em>only</em> to the three
* billable categories, and resets each billing period.
*
* <p>Layout: a slim <b>Editor plan</b> card (always-free tools only — no dates,
* no metered split) on top, then a single <b>Processor plan</b> card that
* two-columns the upgrade pitch + benefits (left) against the one-time free
* meter stacked over the call-to-action (right).
* two-columns the upgrade pitch + benefits (left) against the free-grant meter
* stacked over the call-to-action (right).
*
* <p>Two variants:
* - {@link PaygFreeLeader} — the right column's CTA opens the upgrade modal.
@@ -47,8 +46,6 @@ import {
type FreeSnapshot,
} from "@app/components/shared/config/configSections/usageMeters";
// ─── Editor plan card (always-free tools only) ────────────────────────────
interface EditorPlanCardProps {
/** Role pill text on the right. */
pill: string;
@@ -58,8 +55,8 @@ interface EditorPlanCardProps {
/**
* The top card: the free Editor plan. Manual tools only, no billing window —
* the one-time grant lives in the Processor card below, so there's no period
* to show here.
* the metered grant lives in the Processor card below, so there's no period to
* show here.
*/
function EditorPlanCard({ pill, leader }: EditorPlanCardProps) {
const { t } = useTranslation();
@@ -93,8 +90,6 @@ function EditorPlanCard({ pill, leader }: EditorPlanCardProps) {
);
}
// ─── Processor plan card (two-column: pitch + benefits | meter + CTA) ──────
interface ProcessorCardProps {
snap: FreeSnapshot;
/** Leaders get the live CTA; members get the ask-owner note. */
@@ -207,8 +202,6 @@ function ProcessorCard({ snap, isLeader, onTurnOn }: ProcessorCardProps) {
);
}
// ─── Free LEADER ──────────────────────────────────────────────────────────
export interface PaygFreeLeaderProps {
/**
* Called when the user finishes the {@link UpgradeModal} checkout flow.
@@ -265,8 +258,6 @@ function PaygFreeLeaderInner({ onUpgraded }: PaygFreeLeaderProps = {}) {
);
}
// ─── Free MEMBER ──────────────────────────────────────────────────────────
function PaygFreeMemberInner() {
useRenderCount("PaygFreeMember");
const { t } = useTranslation();
@@ -69,10 +69,9 @@ interface UpgradeModalProps {
/** ISO 4217 currency code for the cap input. Default USD. */
currency?: "USD" | "EUR" | "GBP";
/**
* The team's one-time free grant in documents — the real {@code
* The team's free grant in documents per billing period — the real {@code
* wallet.freeAllowance}, threaded from the free-leader view so the step copy
* quotes the backend's number instead of a hardcoded one. A lifetime grant,
* not a monthly one.
* quotes the backend's number instead of a hardcoded one.
*/
freeLimit: number;
/**
@@ -17,12 +17,10 @@ import {
import "@app/components/shared/config/configSections/Payg.css";
import "@app/components/shared/config/configSections/PaygFree.css";
// ─── One-time free grant meter ──────────────────────────────────────────────
export interface FreeSnapshot {
/** One-time free documents used so far (grant remaining). */
/** Free documents used so far this period (grant remaining). */
billableUsed: number;
/** The team's one-time free grant size in documents. */
/** The team's free grant size in documents, per billing period. */
billableLimit: number;
}
@@ -64,9 +62,11 @@ export function FreeMeterPanel({ snap }: { snap: FreeSnapshot }) {
pct={pct}
barLabel={t("payg.free.hero.barAria", "Free PDFs remaining")}
figure={remaining.toLocaleString()}
capSuffix={t("payg.free.hero.capSuffix", "of {{limit}} free PDFs left", {
limit: snap.billableLimit.toLocaleString(),
})}
capSuffix={t(
"payg.free.hero.capSuffix",
"of {{limit}} free PDFs left this month",
{ limit: snap.billableLimit.toLocaleString() },
)}
statusLabel={stateLabel}
meta={
<span>
@@ -77,8 +77,6 @@ export function FreeMeterPanel({ snap }: { snap: FreeSnapshot }) {
);
}
// ─── Monthly spend-cap meter ────────────────────────────────────────────────
export interface SpendCapSnapshot {
/** Money spent so far this billing period, in major currency units. */
spent: number;
@@ -106,8 +104,8 @@ export function spendCapSnapshotFromWallet(
}
/**
* Sibling of {@link FreeMeterPanel} for the money cap rather than the one-time
* free grant. Shares the same bar/status styling and the cap-state labels
* Sibling of {@link FreeMeterPanel} for the money cap rather than the free
* grant. Shares the same bar/status styling and the cap-state labels
* ({@code payg.state.*}) used by the Plan hero, so it reads as the same meter.
*/
export function SpendCapMeterPanel({ snap }: { snap: SpendCapSnapshot }) {
@@ -149,8 +147,6 @@ export function SpendCapMeterPanel({ snap }: { snap: SpendCapSnapshot }) {
);
}
// ─── Prepaid bundle capacity meter ──────────────────────────────────────────
export interface PrepaidSnapshot {
/** Prepaid units still available across the team's in-term pools. */
remaining: number;
@@ -13,10 +13,8 @@ function toCredits(
freeRemaining: number,
freeAllowance: number,
): CachedCredits {
// Free teams only. The grant is a lifetime pool that survives subscribing, so
// a paying team would otherwise sit on a permanent "0 of 500" in red while
// nothing is wrong. Plan draws the same line — subscribed teams get the
// spend-vs-cap meter there, and admins get usage in the processor.
// Free teams only: a payer's headline number is spend against cap, and a
// draining free meter beside a live invoice reads as a problem.
if (status === "subscribed") return null;
return { remaining: freeRemaining, total: freeAllowance };
}
@@ -4,7 +4,7 @@
*
* <ul>
* <li>{@code 402 FEATURE_DEGRADED} — an authenticated (JWT/web) team hit a
* billable feature it no longer has: a free team that spent its one-time
* billable feature it no longer has: a free team that spent this period's
* allowance, or a subscribed team over its monthly spending cap. Which
* one is told by the {@code subscribed} field on the body.</li>
* <li>{@code 402 PAYG_LIMIT_REACHED} — same situation reached via an API key
@@ -0,0 +1,22 @@
import apiClient from "@app/services/apiClient";
export async function fetchAdminSection<T>(sectionName: string): Promise<T> {
const response = await apiClient.get<T>(
`/api/v1/admin/settings/section/${sectionName}`,
);
return (response.data ?? {}) as T;
}
export async function putAdminSection(
sectionName: string,
delta: unknown,
): Promise<void> {
await apiClient.put(`/api/v1/admin/settings/section/${sectionName}`, delta);
}
/** Flat dotted-path settings, for sections that write outside their own block. */
export async function putAdminSettings(
settings: Record<string, unknown>,
): Promise<void> {
await apiClient.put("/api/v1/admin/settings", { settings });
}
@@ -604,7 +604,10 @@ const FileEditorThumbnail = ({
{/* Badges — top-left: version, pin, ownership, encrypted */}
<div className={styles.thumbBadges}>
<span className={styles.versionBadgeThumb}>
<span
className={styles.versionBadgeThumb}
data-testid="file-version-badge"
>
v{file.versionNumber}
</span>
{isPinned && (
@@ -7,6 +7,7 @@ import ChevronLeftIcon from "@mui/icons-material/ChevronLeft";
import ChevronRightIcon from "@mui/icons-material/ChevronRight";
import { useTranslation } from "react-i18next";
import { getFileSize } from "@app/utils/fileUtils";
import { toolOperationLabel } from "@app/utils/toolOperationLabel";
import { StirlingFileStub } from "@app/types/fileContext";
import { PrivateContent } from "@app/components/shared/PrivateContent";
@@ -115,7 +116,7 @@ const CompactFileDetails: React.FC<CompactFileDetailsProps> = ({
{currentFile?.toolHistory && currentFile.toolHistory.length > 0 && (
<Text size="xs" c="dimmed">
{currentFile.toolHistory
.map((tool) => t(`home.${tool.toolId}.title`, tool.toolId))
.map((tool) => toolOperationLabel(tool, t))
.join(" → ")}
</Text>
)}
@@ -20,7 +20,13 @@ import CreateNewFolderIcon from "@mui/icons-material/CreateNewFolder";
import SearchIcon from "@mui/icons-material/Search";
import { FileId } from "@app/types/file";
import { FolderId, FolderRecord, ROOT_FOLDER_ID } from "@app/types/folder";
import {
FolderId,
FolderRecord,
ROOT_FOLDER_ID,
folderKind,
} from "@app/types/folder";
import type { DiskFileEntry } from "@app/services/localFolderContents";
import { useFolders } from "@app/contexts/FolderContext";
import { usePolicyFileBadges } from "@app/hooks/usePolicyFileBadges";
import { StirlingFileStub } from "@app/types/fileContext";
@@ -36,20 +42,64 @@ import { FileOriginBadge } from "@app/components/filesPage/FileOriginBadge";
import { FolderThumbnail } from "@app/components/filesPage/FolderThumbnail";
import { findFolderIcon } from "@app/components/filesPage/folderIcons";
import { FolderAppearancePicker } from "@app/components/filesPage/FolderAppearancePicker";
import { useLazyThumbnail } from "@app/hooks/useLazyThumbnail";
import {
useLazyThumbnail,
useDiskThumbnail,
} from "@app/hooks/useLazyThumbnail";
import { useFileActionIcons } from "@app/hooks/useFileActionIcons";
import { useFileActionTerminology } from "@app/hooks/useFileActionTerminology";
import type { FilesPageSortMode } from "@app/contexts/FilesPageContext";
import { OpenInNewWindowMenuItem } from "@app/components/filesPage/OpenInNewWindowMenuItem";
/**
* The origin badge a folder wears, mirroring the one its files would: a server folder
* is Cloud, a virtual folder is Local (this browser), a mounted folder is On disk.
*/
function useFolderOriginBadge(folder: FolderRecord): {
origin: "cloud" | "local";
tooltip: string;
} {
const { t } = useTranslation();
switch (folderKind(folder)) {
case "virtual":
return {
origin: "local",
tooltip: t(
"filesPage.folderOrigin.virtualHint",
"A folder that lives only in this browser",
),
};
case "local":
return {
// Same mark as a virtual folder: what matters is that it lives on
// this device, not which corner of it. The tooltip says which.
origin: "local",
tooltip: t(
"filesPage.folderOrigin.diskHint",
"A folder mounted from a directory on your disk",
),
};
default:
return {
origin: "cloud",
tooltip: t(
"filesPage.folderOrigin.serverHint",
"A folder stored on the Stirling server",
),
};
}
}
export type FilesPageViewMode = "grid" | "list";
export interface FilesPageEntry {
kind: "folder" | "file";
kind: "folder" | "file" | "diskFile";
folder?: FolderRecord;
/** Number of files inside this folder (folder entries only). */
folderFileCount?: number;
file?: StirlingFileStub;
/** A file read straight off a mounted directory (kind "diskFile"). */
disk?: DiskFileEntry;
/** Parent breadcrumb path for search results outside the current folder. */
parentPath?: string;
}
@@ -66,6 +116,7 @@ interface FileGridProps {
onOpenFolder: (id: FolderId) => void;
/** "Add to workspace". */
onOpenFile: (file: StirlingFileStub) => void;
onOpenDiskFile?: (entry: DiskFileEntry) => void;
onMoveFiles: (
fileIds: FileId[],
targetFolderId: FolderId | null,
@@ -383,6 +434,7 @@ function GridView(props: FileGridProps) {
onSelectFile,
onOpenFolder,
onOpenFile,
onOpenDiskFile,
onMoveFiles,
onMoveFolder,
onRenameFolder,
@@ -414,6 +466,15 @@ function GridView(props: FileGridProps) {
/>
);
}
if (entry.kind === "diskFile" && entry.disk) {
return (
<DiskFileCard
key={`disk-${entry.disk.path}`}
entry={entry.disk}
onOpen={() => onOpenDiskFile?.(entry.disk!)}
/>
);
}
if (entry.kind === "file" && entry.file) {
return (
<FileCard
@@ -470,6 +531,12 @@ function FolderCard({
}: FolderCardProps) {
const { t } = useTranslation();
const { serverReachable, setError } = useFolders();
// Only a server folder can go offline: the other kinds take their name, look and
// lifetime from elsewhere, so their edit items are hidden rather than disabled.
const kind = folderKind(folder);
const originBadge = useFolderOriginBadge(folder);
const editsDisabled = kind === "server" && !serverReachable;
const editsHidden = kind === "local";
const offlineHint = t(
"filesPage.offlineNoFolderEdits",
"Offline - folder changes are disabled.",
@@ -517,9 +584,7 @@ function FolderCard({
e.dataTransfer.effectAllowed = "move";
}}
{...dropHandlers}
className={`files-page-card is-folder${
isDropTarget ? " is-drop-target" : ""
}`}
className={`files-page-card is-folder${isDropTarget ? " is-drop-target" : ""}`}
onDoubleClick={onOpen}
onContextMenu={(e) => {
e.preventDefault();
@@ -540,6 +605,13 @@ function FolderCard({
fileCount={fileCount}
iconGlyph={findFolderIcon(folder.icon)?.glyph}
/>
<div className="files-page-card-origin">
<FileOriginBadge
origin={originBadge.origin}
tooltip={originBadge.tooltip}
compact
/>
</div>
</div>
<div className="files-page-card-body">
<div className="files-page-card-name" title={folder.name}>
@@ -577,33 +649,51 @@ function FolderCard({
>
{t("filesPage.open", "Open")}
</Menu.Item>
<Menu.Item
leftSection={<DriveFileRenameOutlineIcon fontSize="small" />}
onClick={onRename}
disabled={!serverReachable}
title={!serverReachable ? offlineHint : undefined}
>
{t("filesPage.rename", "Rename")}
</Menu.Item>
<Menu.Divider />
<Menu.Label>
{t("filesPage.appearance.title", "Appearance")}
</Menu.Label>
<FolderAppearancePicker
folder={folder}
onChange={onChangeAppearance}
disabled={!serverReachable}
/>
<Menu.Divider />
<Menu.Item
color="red"
leftSection={<DeleteIcon fontSize="small" />}
onClick={onDelete}
disabled={!serverReachable}
title={!serverReachable ? offlineHint : undefined}
>
{t("filesPage.deleteFolder", "Delete folder")}
</Menu.Item>
{/* Only a mount root can be removed; a subdirectory is the
disk's, and the app never deletes directories. */}
{editsHidden && folder.parentFolderId === null && (
<Menu.Item
color="red"
leftSection={<DeleteIcon fontSize="small" />}
onClick={onDelete}
>
{t(
"filesPage.removeLocalFolder",
"Remove (files stay on disk)",
)}
</Menu.Item>
)}
{!editsHidden && (
<>
<Menu.Item
leftSection={<DriveFileRenameOutlineIcon fontSize="small" />}
onClick={onRename}
disabled={editsDisabled}
title={editsDisabled ? offlineHint : undefined}
>
{t("filesPage.rename", "Rename")}
</Menu.Item>
<Menu.Divider />
<Menu.Label>
{t("filesPage.appearance.title", "Appearance")}
</Menu.Label>
<FolderAppearancePicker
folder={folder}
onChange={onChangeAppearance}
disabled={editsDisabled}
/>
<Menu.Divider />
<Menu.Item
color="red"
leftSection={<DeleteIcon fontSize="small" />}
onClick={onDelete}
disabled={editsDisabled}
title={editsDisabled ? offlineHint : undefined}
>
{t("filesPage.deleteFolder", "Delete folder")}
</Menu.Item>
</>
)}
</Menu.Dropdown>
</Menu>
</div>
@@ -908,9 +998,7 @@ function FileCard({
onKeyDown={(e) => {
if (e.key === "Enter") onDoubleClick();
}}
className={`files-page-card${isSelected ? " is-selected" : ""}${
isInWorkspace ? " is-in-workspace" : ""
}`}
className={`files-page-card${isSelected ? " is-selected" : ""}${isInWorkspace ? " is-in-workspace" : ""}`}
>
{isInWorkspace && (
<span
@@ -1011,6 +1099,7 @@ function ListView(
onSetSelection,
onOpenFolder,
onOpenFile,
onOpenDiskFile,
onMoveFiles,
onMoveFolder,
onRenameFolder,
@@ -1129,6 +1218,15 @@ function ListView(
/>
);
}
if (entry.kind === "diskFile" && entry.disk) {
return (
<DiskFileRow
key={`disk-${entry.disk.path}`}
entry={entry.disk}
onOpen={() => onOpenDiskFile?.(entry.disk!)}
/>
);
}
if (entry.kind === "file" && entry.file) {
return (
<FileRow
@@ -1183,6 +1281,11 @@ function FolderRow({
}: FolderRowProps) {
const { t } = useTranslation();
const { serverReachable, setError } = useFolders();
// Kinds gate the edit items, as in FolderCard.
const kind = folderKind(folder);
const originBadge = useFolderOriginBadge(folder);
const editsDisabled = kind === "server" && !serverReachable;
const editsHidden = kind === "local";
const offlineHint = t(
"filesPage.offlineNoFolderEdits",
"Offline - folder changes are disabled.",
@@ -1278,8 +1381,19 @@ function FolderRow({
</span>
)}
</span>
<FileOriginBadge
origin={originBadge.origin}
tooltip={originBadge.tooltip}
compact
/>
</span>
<span role="gridcell">
{kind === "virtual"
? t("filesPage.folderKind.virtual", "Browser folder")
: kind === "local"
? t("filesPage.folderKind.local", "Local folder")
: t("filesPage.folder", "Folder")}
</span>
<span role="gridcell">{t("filesPage.folder", "Folder")}</span>
<span role="gridcell">
{fileCount === 0
? "-"
@@ -1308,33 +1422,51 @@ function FolderRow({
>
{t("filesPage.open", "Open")}
</Menu.Item>
<Menu.Item
leftSection={<DriveFileRenameOutlineIcon fontSize="small" />}
onClick={onRename}
disabled={!serverReachable}
title={!serverReachable ? offlineHint : undefined}
>
{t("filesPage.rename", "Rename")}
</Menu.Item>
<Menu.Divider />
<Menu.Label>
{t("filesPage.appearance.title", "Appearance")}
</Menu.Label>
<FolderAppearancePicker
folder={folder}
onChange={onChangeAppearance}
disabled={!serverReachable}
/>
<Menu.Divider />
<Menu.Item
color="red"
leftSection={<DeleteIcon fontSize="small" />}
onClick={onDelete}
disabled={!serverReachable}
title={!serverReachable ? offlineHint : undefined}
>
{t("filesPage.deleteFolder", "Delete folder")}
</Menu.Item>
{/* Only a mount root can be removed; a subdirectory is the
disk's, and the app never deletes directories. */}
{editsHidden && folder.parentFolderId === null && (
<Menu.Item
color="red"
leftSection={<DeleteIcon fontSize="small" />}
onClick={onDelete}
>
{t(
"filesPage.removeLocalFolder",
"Remove (files stay on disk)",
)}
</Menu.Item>
)}
{!editsHidden && (
<>
<Menu.Item
leftSection={<DriveFileRenameOutlineIcon fontSize="small" />}
onClick={onRename}
disabled={editsDisabled}
title={editsDisabled ? offlineHint : undefined}
>
{t("filesPage.rename", "Rename")}
</Menu.Item>
<Menu.Divider />
<Menu.Label>
{t("filesPage.appearance.title", "Appearance")}
</Menu.Label>
<FolderAppearancePicker
folder={folder}
onChange={onChangeAppearance}
disabled={editsDisabled}
/>
<Menu.Divider />
<Menu.Item
color="red"
leftSection={<DeleteIcon fontSize="small" />}
onClick={onDelete}
disabled={editsDisabled}
title={editsDisabled ? offlineHint : undefined}
>
{t("filesPage.deleteFolder", "Delete folder")}
</Menu.Item>
</>
)}
</Menu.Dropdown>
</Menu>
</span>
@@ -1402,9 +1534,7 @@ function FileRow({
onKeyDown={(e) => {
if (e.key === "Enter") onOpen();
}}
className={`files-page-list-row${isSelected ? " is-selected" : ""}${
isInWorkspace ? " is-in-workspace" : ""
}`}
className={`files-page-list-row${isSelected ? " is-selected" : ""}${isInWorkspace ? " is-in-workspace" : ""}`}
>
{/* Each direct child is a gridcell: a role="row" may only own cells, so the
checkbox and the actions menu have to sit inside one.
@@ -1516,3 +1646,194 @@ function FileRow({
// Re-export root constant for caller convenience
export { ROOT_FOLDER_ID };
/**
* No stub behind it, so no selection, move, rename or delete: the disk owns the
* file and the only affordance is adding it to the workspace.
*/
function DiskFileCard({
entry,
onOpen,
}: {
entry: DiskFileEntry;
onOpen: () => void;
}) {
const { t } = useTranslation();
const thumbnail = useDiskThumbnail(entry);
const extension = entry.name.includes(".")
? entry.name.split(".").pop()!.toUpperCase()
: "";
const isPdf = extension === "PDF";
return (
<div
className="files-page-card"
role="listitem"
tabIndex={0}
onDoubleClick={onOpen}
onKeyDown={(e) => {
if (e.key === "Enter") onOpen();
}}
title={entry.path}
>
<div className="files-page-card-thumb">
{thumbnail ? (
<img src={thumbnail} alt="" draggable={false} />
) : (
<div className="files-page-card-thumb-fallback">
{isPdf ? (
<PictureAsPdfIcon style={{ fontSize: "2rem" }} />
) : (
<InsertDriveFileIcon style={{ fontSize: "2rem" }} />
)}
<span>{extension || "FILE"}</span>
</div>
)}
<div className="files-page-card-origin">
<FileOriginBadge
origin="local"
tooltip={t(
"filesPage.origin.diskHint",
"A file in the mounted folder on your disk",
)}
compact
/>
</div>
</div>
<div className="files-page-card-body">
<div className="files-page-card-name" title={entry.name}>
{entry.name}
</div>
<div className="files-page-card-meta">
<span>{formatFileSize(entry.sizeBytes)}</span>
<span>·</span>
<span>{getFileDate({ lastModified: entry.lastModified })}</span>
</div>
</div>
<div className="files-page-card-actions">
<Menu shadow="md" position="bottom-end" withinPortal>
<Menu.Target>
<ActionIcon
size="sm"
onClick={(e) => e.stopPropagation()}
aria-label={t("filesPage.fileMenu", "File actions")}
>
<MoreVertIcon fontSize="small" />
</ActionIcon>
</Menu.Target>
<Menu.Dropdown>
<Menu.Item
leftSection={<OpenInNewIcon fontSize="small" />}
onClick={(e) => {
e.stopPropagation();
onOpen();
}}
>
{t("filesPage.addToWorkspace", "Add to workspace")}
</Menu.Item>
</Menu.Dropdown>
</Menu>
</div>
</div>
);
}
/** List-view sibling of {@link DiskFileCard}; same single affordance. */
function DiskFileRow({
entry,
onOpen,
}: {
entry: DiskFileEntry;
onOpen: () => void;
}) {
const { t } = useTranslation();
const thumbnail = useDiskThumbnail(entry);
const ext = entry.name.includes(".")
? entry.name.split(".").pop()!.toUpperCase()
: "";
return (
<div
role="row"
tabIndex={0}
className="files-page-list-row"
onDoubleClick={onOpen}
onKeyDown={(e) => {
if (e.key === "Enter") onOpen();
}}
title={entry.path}
>
<span aria-hidden="true" />
<span
role="gridcell"
style={{
display: "flex",
alignItems: "center",
gap: "0.5rem",
minWidth: 0,
}}
>
{thumbnail ? (
<img
src={thumbnail}
alt=""
draggable={false}
style={{
width: "1.5rem",
height: "1.5rem",
objectFit: "cover",
borderRadius: "0.25rem",
}}
/>
) : ext === "PDF" ? (
<PictureAsPdfIcon fontSize="small" />
) : (
<InsertDriveFileIcon fontSize="small" />
)}
<span
style={{
overflow: "hidden",
textOverflow: "ellipsis",
whiteSpace: "nowrap",
}}
title={entry.name}
>
{entry.name}
</span>
<FileOriginBadge
origin="local"
tooltip={t(
"filesPage.origin.diskHint",
"A file in the mounted folder on your disk",
)}
compact
/>
</span>
<span role="gridcell">{ext || t("filesPage.file", "File")}</span>
<span role="gridcell">{formatFileSize(entry.sizeBytes)}</span>
<span role="gridcell">
{getFileDate({ lastModified: entry.lastModified })}
</span>
<span role="gridcell">
<Menu shadow="md" position="bottom-end" withinPortal>
<Menu.Target>
<ActionIcon
variant="tertiary"
size="sm"
onClick={(e) => e.stopPropagation()}
aria-label={t("filesPage.fileMenu", "File actions")}
>
<MoreVertIcon fontSize="small" />
</ActionIcon>
</Menu.Target>
<Menu.Dropdown>
<Menu.Item
leftSection={<OpenInNewIcon fontSize="small" />}
onClick={onOpen}
>
{t("filesPage.addToWorkspace", "Add to workspace")}
</Menu.Item>
</Menu.Dropdown>
</Menu>
</span>
</div>
);
}
@@ -10,8 +10,10 @@ import { useLocation, useNavigate } from "react-router-dom";
import {
Drawer,
Group,
Menu,
MultiSelect,
Select,
Text,
TextInput,
Tooltip,
} from "@mantine/core";
@@ -32,6 +34,9 @@ import OpenInNewIcon from "@mui/icons-material/OpenInNew";
import InfoOutlinedIcon from "@mui/icons-material/InfoOutlined";
import CloudUploadIcon from "@mui/icons-material/CloudUpload";
import KeyboardArrowRightIcon from "@mui/icons-material/KeyboardArrowRight";
import ArrowDropDownIcon from "@mui/icons-material/ArrowDropDown";
import DriveFolderUploadIcon from "@mui/icons-material/DriveFolderUpload";
import CloudIcon from "@mui/icons-material/Cloud";
import RefreshIcon from "@mui/icons-material/Refresh";
import { FilesToolbarBulkMenu } from "@app/components/filesPage/FilesToolbarBulkMenu";
import { FilesToolbarCount } from "@app/components/filesPage/FilesToolbarCount";
@@ -45,6 +50,7 @@ import { useFolders } from "@app/contexts/FolderContext";
import { useFileActions } from "@app/contexts/file/fileHooks";
import { useAllFiles } from "@app/contexts/FileContext";
import { useFileHandler } from "@app/hooks/useFileHandler";
import { useServerFolderBlock } from "@app/hooks/useServerFolderBlock";
import {
useNavigationActions,
useNavigationGuard,
@@ -60,7 +66,7 @@ import { getFileOrigin } from "@app/components/filesPage/fileOrigin";
import { FileId } from "@app/types/file";
import { StirlingFileStub } from "@app/types/fileContext";
import { FolderId, ROOT_FOLDER_ID } from "@app/types/folder";
import { FolderId, ROOT_FOLDER_ID, folderKind } from "@app/types/folder";
import { FileGrid, FilesPageEntry } from "@app/components/filesPage/FileGrid";
import SuperSearch from "@app/components/shared/superSearch/SuperSearch";
@@ -69,6 +75,20 @@ import { FileDetailsPanel } from "@app/components/filesPage/FileDetailsPanel";
import BulkUploadToServerModal from "@app/components/shared/BulkUploadToServerModal";
import MobileUploadModal from "@app/components/shared/MobileUploadModal";
import { useAppConfig } from "@app/contexts/AppConfigContext";
import { canPickDirectory } from "@app/services/directoryPicker";
import {
diskFolderId,
isDiskFolderId,
pickFolderColor,
} from "@app/types/folder";
import { useNewFolderFlow } from "@app/hooks/useNewFolderFlow";
import { writeIntoMount } from "@app/services/mountWrites";
import {
canListDirectory,
listDirectory,
readDiskFile,
type DiskFileEntry,
} from "@app/services/localFolderContents";
import { useIsMobile } from "@app/hooks/useIsMobile";
import { MoveToFolderDialog } from "@app/components/filesPage/MoveToFolderDialog";
import { FolderNameDialog } from "@app/components/filesPage/FolderNameDialog";
@@ -201,6 +221,7 @@ export default function FileManagerView() {
);
const setCurrentFolderId = folders.setCurrentFolderId;
const resolveDiskFolder = folders.resolveDiskFolder;
const foldersById = folders.foldersById;
const currentFolderId = folders.currentFolderId;
@@ -212,10 +233,13 @@ export default function FileManagerView() {
setCurrentFolderId(ROOT_FOLDER_ID);
} else if (foldersById.has(param as FolderId)) {
setCurrentFolderId(param as FolderId);
} else if (isDiskFolderId(param) && resolveDiskFolder(param as FolderId)) {
// A mount subdirectory deep link: rebuilt from the id, mapped next render.
setCurrentFolderId(param as FolderId);
} else {
setCurrentFolderId(ROOT_FOLDER_ID);
}
}, [location.pathname, foldersById, setCurrentFolderId]);
}, [location.pathname, foldersById, setCurrentFolderId, resolveDiskFolder]);
// Bounce off any share-related tab when sharing isn't enabled.
useEffect(() => {
@@ -275,6 +299,8 @@ export default function FileManagerView() {
}
const lc = search.toLowerCase();
const matched = folders.folders.filter((f) => {
// The Cloud tab is the server's view: browser folders and mounts aren't on it.
if (currentTab === "cloud" && folderKind(f) !== "server") return false;
if (search) {
// Subtree-wide name match; exclude the current folder itself.
return (
@@ -296,10 +322,11 @@ export default function FileManagerView() {
// Tab overrides folder navigation for Local/Recent/Shared.
switch (currentTab) {
case "local":
// Local = files with no server copy. folderId is forced null on this
// path (cf. file.ts comment), but we check remoteStorageId too so
// stale local-folder rows from a pre-pivot DB don't slip through.
return allFiles.filter((f) => f.remoteStorageId == null);
// Both halves: a local file inside a browser folder belongs to that folder,
// not here as well.
return allFiles.filter(
(f) => f.remoteStorageId == null && (f.folderId ?? null) === null,
);
case "cloud":
// Cloud bucket; search widens to subtree, else direct-folder match.
return allFiles.filter((f) => {
@@ -426,12 +453,141 @@ export default function FileManagerView() {
[foldersById],
);
const currentFolder = currentFolderId
? folders.foldersById.get(currentFolderId)
: undefined;
const currentLocalDirectory =
currentFolder && folderKind(currentFolder) === "local"
? currentFolder.directory
: undefined;
const { setError: setFolderError, registerDiskSubfolders } = folders;
const [diskEntries, setDiskEntries] = useState<DiskFileEntry[]>([]);
const [diskLoading, setDiskLoading] = useState(false);
// Bumped when this view writes into the directory, so the listing re-reads.
const [diskRefreshTick, setDiskRefreshTick] = useState(0);
useEffect(() => {
if (!currentLocalDirectory || !canListDirectory) {
setDiskEntries([]);
// Leaving a mount mid-listing cancels the in-flight reset, so clear the
// flag here or the skeleton covers every folder for the rest of the session.
setDiskLoading(false);
return;
}
let cancelled = false;
setDiskLoading(true);
listDirectory(currentLocalDirectory)
.then((listed) => {
if (cancelled) return;
setDiskEntries(listed?.files ?? []);
if (currentFolderId !== null) {
registerDiskSubfolders(
currentFolderId,
(listed?.directories ?? []).map((dir) => ({
id: diskFolderId(dir.path),
kind: "local" as const,
name: dir.name,
parentFolderId: currentFolderId,
directory: dir.path,
color: pickFolderColor(dir.name),
createdAt: 0,
updatedAt: 0,
})),
);
}
})
.catch((err) => {
console.warn("[FileManagerView] disk listing failed", err);
if (!cancelled) {
setDiskEntries([]);
setFolderError(
err instanceof Error
? t("filesPage.error.readFolderFailedDetail", {
message: err.message,
defaultValue: `Could not read the folder: ${err.message}`,
})
: t(
"filesPage.error.readFolderFailed",
"Could not read the folder.",
),
);
}
})
.finally(() => {
if (!cancelled) setDiskLoading(false);
});
return () => {
cancelled = true;
};
// The stable setter, not the context: its identity changes on every folder
// mutation, including the setError above, so a failing listing would re-trigger.
}, [
currentLocalDirectory,
currentFolderId,
registerDiskSubfolders,
setFolderError,
diskRefreshTick,
t,
]);
const openDiskFile = useCallback(
async (entry: DiskFileEntry) => {
try {
const file = await readDiskFile(entry);
if (!file) return;
clearFilesPageReturnRoute();
await addFiles([file], { selectFiles: true });
navActions.setWorkbench("viewer");
navigate("/");
} catch (err) {
folders.setError(
err instanceof Error
? t("filesPage.error.openDiskFileFailedDetail", {
name: entry.name,
message: err.message,
defaultValue: `Could not open ${entry.name}: ${err.message}`,
})
: t("filesPage.error.openDiskFileFailed", {
name: entry.name,
defaultValue: `Could not open ${entry.name}.`,
}),
);
}
},
[addFiles, navActions, navigate, folders, t],
);
const entries = useMemo<FilesPageEntry[]>(() => {
// When searching, items may come from anywhere in the subtree, so we
// expose a "parentPath" subtitle whenever the item's parent differs from
// currentFolderId. When no search is active, every item is in the
// current folder by definition and the subtitle is suppressed.
const inSearch = search.length > 0;
// Inside a mount the listing is the directory; storage rows don't apply.
if (currentLocalDirectory) {
const needle = search.toLowerCase();
const compare: Record<
string,
(a: DiskFileEntry, b: DiskFileEntry) => number
> = {
"name-asc": (a, b) => a.name.localeCompare(b.name),
"name-desc": (a, b) => b.name.localeCompare(a.name),
"size-asc": (a, b) => a.sizeBytes - b.sizeBytes,
"size-desc": (a, b) => b.sizeBytes - a.sizeBytes,
"modified-asc": (a, b) => a.lastModified - b.lastModified,
"modified-desc": (a, b) => b.lastModified - a.lastModified,
};
return [
...visibleFolders.map<FilesPageEntry>((folder) => ({
kind: "folder",
folder,
folderFileCount: 0,
})),
...diskEntries
.filter((disk) => !needle || disk.name.toLowerCase().includes(needle))
.sort(compare[filesPage.sortMode] ?? compare["modified-desc"]!)
.map<FilesPageEntry>((disk) => ({ kind: "diskFile", disk })),
];
}
return [
...visibleFolders.map<FilesPageEntry>((folder) => ({
kind: "folder",
@@ -457,6 +613,9 @@ export default function FileManagerView() {
filesPage.fileCountsByFolder,
search,
currentFolderId,
currentLocalDirectory,
diskEntries,
filesPage.sortMode,
pathForFolderId,
]);
@@ -531,28 +690,46 @@ export default function FileManagerView() {
// state - otherwise the file pops up the next time the user navigates
// to /viewer or /tools, which reads as "auto-opened" and surprised
// people every time. The grid will repaint via refresh() below.
// Files uploaded while standing in a folder belong in that folder.
const target =
currentTab === "all" || currentTab === "cloud" ? currentFolderId : null;
const targetFolder = target ? folders.foldersById.get(target) : undefined;
if (targetFolder && folderKind(targetFolder) === "local") {
const { failedCount } = await writeIntoMount(
targetFolder.directory,
files.map((file) => ({ name: file.name, bytes: async () => file })),
);
if (failedCount > 0) {
folders.setError(
t("filesPage.moveIntoMountFailed", {
count: failedCount,
defaultValue:
"{{count}} file(s) could not be written into the folder.",
}),
);
}
setDiskRefreshTick((tick) => tick + 1);
return;
}
// Everywhere else membership is set with the stub rather than by a move that
// could fail after. For a server folder it stays local until the save lands.
const added = await addFiles(files, {
selectFiles: false,
skipWorkspaceDispatch: true,
...(target ? { folderId: target as string } : {}),
});
const fileIds = added.map((f) => f.fileId);
const target = currentFolderId;
// Uploaded files land in Local (folderId stays null).
if (
target !== null &&
fileIds.length > 0 &&
(currentTab === "all" || currentTab === "cloud")
targetFolder &&
folderKind(targetFolder) === "server"
) {
folders.setError(
t(
"filesPage.uploadedToLocal",
"Uploaded files start in Local. Use 'Save to cloud' to put them in a folder.",
),
);
await moveFilesTo(fileIds, target);
}
await refresh();
},
[addFiles, currentFolderId, currentTab, folders, refresh, t],
[addFiles, currentFolderId, currentTab, folders, moveFilesTo, refresh, t],
);
const onFileInputChange = useCallback(
@@ -913,20 +1090,18 @@ export default function FileManagerView() {
[selectedFiles, fileMap],
);
// Per-destination availability for the New-folder menu; the reason renders as the
// disabled item's caption.
const serverFolderDisabledReason = useServerFolderBlock() ?? undefined;
const { addLocalFolder, createFolderHere, createFolderHereBlockedReason } =
useNewFolderFlow();
// null = New folder actionable; string = disabled tooltip reason.
const newFolderDisabledReason: string | null = useMemo(() => {
// Guests can't use cloud folders at all - say so before any tab/storage
// hint, since switching tabs wouldn't help them.
if (signInRequiredReason) {
return signInRequiredReason;
}
if (currentTab === "local") {
return t(
"filesPage.localFoldersUnavailable",
"Folders are cloud-only - save a file to the cloud to organise it.",
);
}
// Only All/Cloud render folders, so creating one elsewhere would look inert.
if (
currentTab === "local" ||
currentTab === "recent" ||
currentTab === "shared" ||
currentTab === "sharedByMe"
@@ -936,14 +1111,32 @@ export default function FileManagerView() {
"Switch to All or Cloud to create folders.",
);
}
if (!folders.serverReachable) {
return t(
"filesPage.newFolderStorageDisabled",
"Server folder storage isn't enabled. Ask your admin to turn it on.",
);
// A subfolder inherits kind server, so the blockers gate the button rather
// than letting the dialog open and fail at submit.
if (
currentFolder &&
folderKind(currentFolder) === "server" &&
serverFolderDisabledReason
) {
return serverFolderDisabledReason;
}
// The web root creates on the server or not at all.
if (
folders.currentFolderId === null &&
!canPickDirectory &&
serverFolderDisabledReason
) {
return serverFolderDisabledReason;
}
return null;
}, [signInRequiredReason, currentTab, folders.serverReachable, t]);
}, [
currentTab,
currentLocalDirectory,
currentFolder,
folders.currentFolderId,
serverFolderDisabledReason,
t,
]);
return (
<div className="files-page" ref={dropZoneRef}>
@@ -977,6 +1170,10 @@ export default function FileManagerView() {
const handleRefresh = async () => {
setRefreshing(true);
try {
// In a mount, refresh means the directory: the listing only re-reads when told.
if (currentLocalDirectory) {
setDiskRefreshTick((tick) => tick + 1);
}
// pullFromServer bumps the folder revision, which the
// FolderProvider's effect reacts to by re-running refresh() -
// no need to await folders.refresh() manually.
@@ -1045,15 +1242,69 @@ export default function FileManagerView() {
</Button>
</span>
</Tooltip>
) : (
) : folders.currentFolderId !== null || !canPickDirectory ? (
// Nothing to choose: a subfolder inherits its parent's kind, and on
// the web everything lives on the server. Straight to the dialog.
<Button
variant="secondary"
size="sm"
leftSection={<CreateNewFolderIcon fontSize="small" />}
onClick={() => openNewFolderDialog()}
onClick={() =>
folders.currentFolderId !== null
? openNewFolderDialog()
: openNewFolderDialog(null, "server")
}
>
{t("filesPage.newFolder", "New folder")}
</Button>
) : (
// Desktop root: two peer destinations, so the button is the menu.
<Menu shadow="md" position="bottom-end" withinPortal>
<Menu.Target>
<Button
variant="secondary"
size="sm"
leftSection={<CreateNewFolderIcon fontSize="small" />}
rightSection={<ArrowDropDownIcon fontSize="small" />}
>
{t("filesPage.newFolder", "New folder")}
</Button>
</Menu.Target>
<Menu.Dropdown>
<Menu.Item
leftSection={
<DriveFolderUploadIcon
fontSize="small"
style={{ marginRight: "0.3rem" }}
/>
}
onClick={() => void addLocalFolder()}
>
{t(
"filesPage.newFolderMenu.addExisting",
"Add local folder",
)}
</Menu.Item>
<Menu.Item
className="files-page-new-folder-option"
leftSection={<CloudIcon fontSize="small" />}
disabled={Boolean(serverFolderDisabledReason)}
onClick={() => openNewFolderDialog(null, "server")}
>
{t(
"filesPage.newFolderMenu.server",
"New folder on the server",
)}
<Text size="xs" c="dimmed">
{serverFolderDisabledReason ??
t(
"filesPage.newFolderMenu.serverHint",
"Synced to your account, available wherever you sign in.",
)}
</Text>
</Menu.Item>
</Menu.Dropdown>
</Menu>
)}
<Button
size="sm"
@@ -1647,7 +1898,7 @@ export default function FileManagerView() {
>
<FileGrid
entries={entries}
loading={loading}
loading={loading || diskLoading}
currentTab={currentTab}
searchActive={search.trim().length > 0}
serverReachable={folders.serverReachable}
@@ -1659,6 +1910,7 @@ export default function FileManagerView() {
onSelectFile={handleSelectFile}
onSetSelection={setSelectedFileIds}
onOpenFolder={handleOpenFolder}
onOpenDiskFile={(entry) => void openDiskFile(entry)}
onOpenFile={handleOpenFile}
onMoveFiles={moveFilesTo}
onMoveFolder={moveFolderTo}
@@ -1692,8 +1944,12 @@ export default function FileManagerView() {
// (disabled tooltips, native file picker, dialog) is
// identical regardless of where the user clicks from.
onEmptyUpload={() => fileInputRef.current?.click()}
onEmptyCreateFolder={() => openNewFolderDialog()}
newFolderDisabledReason={newFolderDisabledReason}
onEmptyCreateFolder={createFolderHere}
// A single-click shortcut, so it also blocks where it has nothing safe
// to do, unlike the header button whose menu still offers the choices.
newFolderDisabledReason={
newFolderDisabledReason ?? createFolderHereBlockedReason
}
/>
{isDraggingExternal && (
<div className="files-page-drop-overlay" aria-live="polite">
@@ -1704,16 +1960,21 @@ export default function FileManagerView() {
{t("filesPage.dropOverlay", "Drop files to upload")}
</span>
<span className="files-page-drop-overlay-sub">
{/* Behavior contract: per handleNativeUpload above, all
newly-uploaded files start in Local (folderId stays
null) regardless of the current folder view. Saying
"will land in {folder}" was a lie; tell the truth
so the user reaches for Save-to-cloud / Move-to when
they actually want a folder placement. */}
{t(
"filesPage.dropOverlaySub",
"Files start in Local. Use 'Move to' or 'Save to cloud' to organise them into a folder.",
)}
{/* Behavior contract: per handleNativeUpload above, files
dropped inside a folder on the All/Cloud views are
placed into it — a mount takes them onto the disk
itself. Other tabs land drops in Local, so the copy
must match. */}
{(currentTab === "all" || currentTab === "cloud") &&
currentFolderId !== null
? t(
"filesPage.dropOverlaySubFolder",
"They'll be added to this folder.",
)
: t(
"filesPage.dropOverlaySub",
"Files land in Local. Organise them into folders any time.",
)}
</span>
</div>
)}
@@ -1774,7 +2035,14 @@ export default function FileManagerView() {
<MoveToFolderDialog
opened={moveDialog.open}
onClose={closeMoveDialog}
folders={folders.folders}
// Files can go anywhere, but a folder moves only within its own kind and
// never into a mount - a directory's subfolders are the filesystem's.
folders={folders.folders.filter((candidate) => {
if (!moveDialog.folderId) return true;
if (folderKind(candidate) === "local") return false;
const moving = folders.foldersById.get(moveDialog.folderId);
return moving ? folderKind(candidate) === folderKind(moving) : true;
})}
initialFolderId={moveDialog.initial}
disabledFolderId={moveDialog.folderId}
onConfirm={async (target) => {

Some files were not shown because too many files have changed in this diff Show More