Commit Graph
100 Commits
Author SHA1 Message Date
James Brunton bbd4d2c3ac Redesign toml sorting to speed up from ~40s to ~2s (#7192)
# Description of Changes
The `pre-commit` tool to sort the translations is really slow. It took
~40 seconds to run because it's using a parser which attempts to save
all of the formatting data from the Toml. Our translations toml is
pretty much entirely formatted anyway, so there's no point in trying to
preserve any of that data. The only thing we lose is 5 comments, none of
which are needed anyway and only appear in the US translation file. By
switching to Python stdlib `tomllib` reading and `tomli-w` for writing,
we can make the Toml formatting job take 2.11 seconds, where it used to
take 39.78s. The whole pre-commit job now takes 4.58 seconds.
2026-07-29 14:33:34 +00:00
James Brunton 999b5e5995 Add persistent outputs to Processor (#7071)
# Description of Changes
<img width="1270" height="487" alt="image"
src="https://github.com/user-attachments/assets/64894e2b-aab9-42ab-96c2-2c11ba427b52"
/>

Change policies to point towards a source for its output instead of a
dynamically defined output location for the pipeline. This allows for
easy reuse of outputs in different pipelines and makes it impossible to
break complex pipelines by accidentally updating the source but not the
output and vice versa. Also makes outputs a list to match the inputs, so
it's possible for a pipeline to output to multiple locations.

We should consider whether we want to continue calling these Sources
since they're now being used as both inputs and outputs, but that
decision is beyond the scope of this PR.

Also updates the existing S3 DB migration script and adds a new one to
migrate to the new schema. Neither of these scripts are possible with
SQL since it involves parsing and restructuring JSON. I've updated them
so that they only ever run once on startup and mark themselves as
completed.
2026-07-29 11:22:47 +00:00
James Brunton 4fbb2fe885 Add PR Quiz skill (#7137)
# Description of Changes
Adds a Claude skill to quiz you about your PR, to help check that you
understand the code in the PR.

<img width="796" height="589" alt="image"
src="https://github.com/user-attachments/assets/0f520a99-9acf-4922-85fe-5f49f8f68823"
/>

<img width="811" height="621" alt="image"
src="https://github.com/user-attachments/assets/e440c775-d901-443b-a213-1cc4ee6303af"
/>
2026-07-24 09:37:10 +00:00
James Brunton 29002d0b82 Improve UX around source folders in Processor (#7101)
# Description of Changes
Adds implicitly defined folders to the list of locations that folder
sources can look in, including the legacy watchedFolder folders, and the
server storage location (if enabled). Also adds a settings UI for
defining the list of allowed folders instead of having to manually edit
`settings.yml` (please excuse the styling, that's the standard styling
of the Processor, hoping it gets fixed by one of the styling PRs).

<img width="888" height="786" alt="image"
src="https://github.com/user-attachments/assets/cf6d0705-adcf-463c-8e80-6901a652068b"
/>

<img width="1103" height="713" alt="image"
src="https://github.com/user-attachments/assets/b7244592-249a-4149-994f-3a2b750f25f9"
/>
2026-07-23 11:19:24 +00:00
James Brunton 5e3e89ccb2 Fix existing teams logic (#7070)
# Description of Changes
Paired with https://github.com/Stirling-Tools/Stirling-PDF-SaaS/pull/320

Fixes the following bugs we found when testing the SaaS release:
- Existing users couldn't join teams - this was because they were the
last leader of their team, so it'd be left orphaned). Users now have a
'home team', which can have no members if they join another team, but
they can then go back to it later.
- Existing leaders didn't have unlimited seats - `saas_teams_extensions`
had no row for them, so the app fell back to `max_seats=1`. The
migration script fixes it.
- Members without Processor access could still access the Processor - It
was just checking "Are you the leader of **any** team", instead of the
user's active team.
2026-07-21 13:57:45 +00:00
James BruntonandEthanHealy01 0fc2958daa Add explicit length to member column to avoid ddl resizing (#7109)
# Description of Changes
There's currently a column size inconsistency between the SaaS v3 DB and
the main Java code which causes the backend to fail to start up when
connected to a fresh DB. This is because the column previously was width
255, but now it's officially width 50, but the Java type is still
implicitly `varchar(255)` because there's no length attribute. If it's a
fresh DB, Postgres throws an error that it can't expand the column (this
doesn't error on an existing DB because the column is already wide
enough behind the scenes).

Co-authored-by: EthanHealy01 <80844253+EthanHealy01@users.noreply.github.com>
2026-07-20 15:23:11 +00:00
James Brunton ecddce9944 Remove agent builder from processor (#7063)
# Description of Changes
Remove agent builder from processor because we don't intend to implement
it any time soon.
2026-07-17 13:15:35 +00:00
James Brunton ed58d90ab8 Remove policies feature flag (#7031)
# Description of Changes
Removes the feature flags for enabling policies on both the backend and
frontend. We shouldn't be releasing another self-hosted release that
doesn't include policies, so it makes sense to do this now. Builds that
don't have the Processor will just not run policies because they won't
have any. Beyond that, the API should always be available, but checks
whether the user actually has the entitlements to run policies (whether
they have credits/a payment method available)
2026-07-15 14:25:25 +00:00
James Brunton 4ee54243f3 Remove encryption from stored policies JSON (#7035)
# Description of Changes
The policies stored in the DB are currently encrypted at rest, because
one version in the past included S3 keys. These are now stored properly
in the credentials system and I've manually removed the only policy that
used S3 (it was very recently released). Since there's no S3 (or other)
credentials in the policies stored JSON now, we might as well just
decrypt them. This PR pairs with #7034 to fix the issues - #7034 makes
it resilient to crashing when attempting to load encrypted JSON that's
been encrypted with the wrong key, and this makes it so if it does load
any encrypted policies, they'll be re-saved decrypted, so we should have
a vanishingly small number of encrypted policies over time.
2026-07-15 12:28:29 +00:00
James Brunton 350c0b796e Upgrade to TS7 official release (#6958)
# Description of Changes
Convert from the TS7 release candidate to the TS7 official release,
keeping TS6 around in [the compatibility mode suggested by
Microsoft](https://devblogs.microsoft.com/typescript/announcing-typescript-7-0/#running-side-by-side-with-typescript-6.0)
so ESLint and our scripts which rely on the TS API still work.
2026-07-15 12:21:34 +00:00
James Brunton 776749277c Redesign policies to use typed mappings properly (#7017)
# Description of Changes
The Policies page and all the frontend logic for running Policies is not
making use of the bidirectional type mappings that we now have to safely
convert from frontend to backend param models and vice versa. This
changes the way we track the types throughout so we use the mappings
properly.

Because of this, the Add Watermark settings in Policies now actually
pre-populate with the defaults instead of with nothing like they
previously did.

<img width="791" height="725" alt="image"
src="https://github.com/user-attachments/assets/cbdf4ae0-35af-4792-bf64-89216e48d304"
/>
2026-07-14 09:58:04 +00:00
James Brunton 41b1b89fcb Fix Policies page showing the Editor as a source twice (#7022)
# Description of Changes
The Policies page currently hard-codes the Editor to be available as a
source, but we now also have a virtual Editor source on the backend,
which the Policies page also renders. This removes the now-unnecessary
hard-coded Editor source.

## Before

<img width="842" height="640" alt="image"
src="https://github.com/user-attachments/assets/d78b33a3-fed4-4bb0-a02f-489ca2ae0614"
/>

## After

<img width="785" height="586" alt="image"
src="https://github.com/user-attachments/assets/37aa664f-74f2-41c7-b89a-9b483bffc3a2"
/>
2026-07-14 09:43:07 +00:00
James Brunton 4d4e994562 Fix crash in Processor when loading tool settings with tooltips (#7015)
# Description of Changes
Some of the tool settings make use of editor preferences indirectly, but
the Processor never gets that provider, so it crashes when trying to
load them.
2026-07-14 09:09:23 +00:00
James Brunton 76549288a9 Redesign S3 connections to use connection resolver (#6965)
# Description of Changes
Redesign S3 connections based on feedback from #6948. Also redesigns the
UI for Sources to make them more like the Pipelines page which improves
UX quite a bit. There's still plenty more UI/UX work for Sources and S3
but moving in the right direction.
2026-07-13 14:44:41 +00:00
James Brunton b019f9b570 Fix missing and broken translations in Processor (#7016)
# Description of Changes
<img width="385" height="92" alt="image"
src="https://github.com/user-attachments/assets/7f4b1921-72e8-4c18-a4de-4b9736a5a2f1"
/>

Started from trying to fix this, but became a larger piece of work to
find missing/broken translations in the Processor and fix as many as I
could.
2026-07-13 12:55:36 +00:00
James Brunton 38d06d3104 Make sqlite backend more resilient when using multiple runners (#6971) 2026-07-11 12:47:29 +01:00
James Brunton 863cad22bd Fix policy running of Redact (#6972)
# Description of Changes
Policies can currently throw when calling redact:

<img width="1186" height="824" alt="image"
src="https://github.com/user-attachments/assets/bdcc09fe-5bf4-4b0a-b119-bcc33c98c7f2"
/>

Policies really need to be updated to properly make use of the new
bidirectional mappings for this, but this will hopefully fix it for now.
2026-07-10 16:24:25 +00:00
James Brunton 84d4455682 Add virtual Editor source (#6959)
# Description of Changes
Adds Editor source permanently available in the Sources list. Excludes
it from the Pipelines list of available sources currently because it's
not a real source on the backend, so attempting to connect to it causes
an error. It'd be nice to extend in the future to be able to set up
policies in the editor from the pipelines page, but this'll do for now.
2026-07-10 13:27:40 +00:00
James Brunton 5ccb56da2d Add S3 policy source (#6948)
# Description of Changes
* Adds an Amazon S3 Source & Output
* Removes folder source from SaaS
* Some miscellaneous UX fixes around pipelines
2026-07-10 12:19:41 +00:00
James Brunton a5ee329c36 Further improvements to policies file tracking (#6941)
# Description of Changes
Fixes requested in review of #6903
2026-07-09 15:43:38 +00:00
James Brunton 01751bf2f0 Improve logic for tracking which files have already been processed in policies (#6903)
# Description of Changes
Replaces the `.stirling/done` folder and its friends with a ledger in
the DB which tracks which documents have been processed. This should
scale dramatically better since it's just a few bytes being written for
each PDF processed, rather than each PDF being duplicated and held in
the folder forever. It's designed to work with the current folder
source, but also with S3 buckets and other sources in mind - each source
will define its own strategy for ensuring it knows whether the documents
have had policies run on them or not, and they all get written to the
same ledger.
2026-07-09 12:07:26 +00:00
James Brunton f29500c138 Disable Portal UI for guests (#6936)
# Description of Changes
Disallow SaaS guests from accessing the portal. One day we might want to
make this better so they can go there but then have to sign up before
doing anything useful, but this is the easiest way to disallow it for
now.
2026-07-09 11:44:19 +00:00
James Brunton d6061eb0aa Support tool selection in Pipelines page in Portal (#6905)
# Description of Changes

Redesigned the Portal Pipelines page so pipelines are created and edited
on their own dedicated builder page, replacing the previous modal
composer and inline detail card.

## Screenshots

### Pipelines list
The redesigned list with summary KPIs; each row opens that pipeline's
own page.

![Pipelines
list](https://raw.githubusercontent.com/Stirling-Tools/Stirling-PDF/pr-6905-screenshots/screenshots/01-pipelines-list.png)

### Pipeline builder
The dedicated create page: pipeline settings (sources, trigger, output)
above, operations and per-tool settings below.

![Pipeline
builder](https://raw.githubusercontent.com/Stirling-Tools/Stirling-PDF/pr-6905-screenshots/screenshots/02-builder-new.png)

### Tool picker
Type-to-filter, category-grouped picker for adding an operation to the
pipeline.

![Tool
picker](https://raw.githubusercontent.com/Stirling-Tools/Stirling-PDF/pr-6905-screenshots/screenshots/03-tool-picker.png)

### Editing a pipeline
An existing pipeline in the builder: reorderable steps, per-tool
settings, and run/delete actions.

![Editing a
pipeline](https://raw.githubusercontent.com/Stirling-Tools/Stirling-PDF/pr-6905-screenshots/screenshots/04-builder-edit.png)
2026-07-08 15:23:04 +00:00
James Brunton 8150d16b6f Support bidirectional mapping for Change Metadata (#6906)
# Description of Changes
The Change Metadata tool was missed from the bidirectional mappings
added in #6867. This PR adds it to the list of supported tools.
2026-07-08 12:47:11 +00:00
James Brunton be57f11747 Improve type safety of tool definitions (#6895)
# Description of Changes
Followup work requested in review of #6867. Currently, there is nothing
enforcing that the endpoint chosen in the tool config is the correct
mapping for `toApiParams`, so theoretically it's possible for a tool to
be set up to call an endpoint with the wrong API params for it. There's
also nothing currently enforcing that `toApiParams` and `fromApiParams`
are compatible with each other (using the same types). This PR changes
it so that instead of creating the config object directly, tools create
it via a generic function, which enforces that all of the relevant
mappings are using compatible types.
2026-07-07 16:43:06 +00:00
James Brunton 17aa71850c Convert to consistently use JS modules (#6854)
# Description of Changes
Modernises the codebase and gets rid of warnings where Node complains
that it doesn't know what type of JS it's supposed to be reading on
`.js` files. We might as well update everything to just use correct JS
syntax instead of keeping with some files having Node-specific imports.
2026-07-07 11:11:24 +00:00
James Brunton 1b7ffcdbac Fix tooltip positioning on Add Page Numbers (#6885)
# Description of Changes
## Before

<img width="483" height="227" alt="image"
src="https://github.com/user-attachments/assets/4bf86eec-a9cc-4f63-84f0-4eb2bd535bab"
/>

## After

<img width="732" height="235" alt="image"
src="https://github.com/user-attachments/assets/101d2ea4-36e8-4e8f-990a-d72b33fa0ac2"
/>
2026-07-07 12:09:51 +01:00
James Brunton 20204f0ddc Improve consistency and reliability of tools in Stirling Engine (#6855)
# Description of Changes
A few changes to improve things in the engine:
- Changed the PDF to Markdown code to be a real tool in Java, to remove
the need for the `pdf_ingest` code, which looked a bit like an agent but
wasn't behaving as an agent. It's now just covered automatically by the
edit agent.
- Noticed that 0-parameter-endpoints were previously being ignored by
the `tool_models` generator, so some tools which require no params were
being mistakenly excluded.
- Removed tools which currently never succeed like Add Stamp, Cert Sign,
and Overlay, because they require the supporting files to be sent in a
different location in the API call, which we don't currently do.
Ideally, we'd add proper support for this, but we're better off now
removing support for these tools rather than just have them crash. We
can re-add these tools in a future PR properly.
2026-07-07 11:01:18 +00:00
James Brunton b4f7b1d8a9 Add bidirectional API types to frontend (#6867)
# Description of Changes
Fix https://github.com/Stirling-Tools/Stirling-PDF-SaaS/issues/281. Add
generated backend API mappings to the frontend code, and the logic to
convert from a backend API to frontend parameters objects.

Previously, it was impossible to tell if changing the backend API would
require a change to the frontend to support it because the frontend had
no static type information about the backend API. This PR adds
autogenerated tool API types to the frontend (in `toolApiTypes.ts`) and
adds explicit typed mappings between the frontend parameter types and
the backend API types, so theoretically the type checker should be able
to catch issues when changing one puts us in an invalid state with the
other. During development, it pointed out several inconsistencies that
we have between the frontend and backend types, some of which were
genuine bugs, and others were only happening to work because the backend
is more permissive than its API claims to be.

This also unlocks the ability for us to render the frontend settings on
saved backend API structures, which we've previously had to avoid doing
because we had no reverse mapping.
2026-07-07 07:47:07 +00:00
James Brunton f881828cd8 Fix intermittently failing Playwright tests (#6886)
# Description of Changes
Fixes intermittently failing tests (and replaces one that wasn't useful
in its previous state) and also adds a CI check to warn if there are any
Playwright tests which failed on their first go and succeeded on
retries, to hopefully help find intermittently failing tests more
quickly and avoid them being merged in the first place.
2026-07-06 21:37:21 +00:00
James Brunton 355d736487 Skip enterprise tests for forks since they can't run without secrets (#6888)
# Description of Changes
OSS contributions which trigger the enterprise e2e tests will always
fail due to missing secrets (see
https://github.com/Stirling-Tools/Stirling-PDF/actions/runs/28776342146/job/85337748018?pr=6045).
This PR disables them for OSS PRs.
2026-07-06 13:26:10 +00:00
James Brunton b69b787d63 Fix Playwright tests in Firefox and Safari (#6868)
# Description of Changes
Playwright tests currently fail in Firefox and Safari because of
inconsistent behaviour across the browsers. This is causing the
nightlies to fail every night. This PR fixes the test behaviour to work
consistently across browsers (most of the issues were to do with the
tests opening the file picker, which was being automatically suppressed
in Chromium, but not the other browsers).
2026-07-06 09:25:46 +00:00
James Brunton e630d6697b Fix tooltip positioning on Add Page Numbers (#6885)
# Description of Changes
## Before

<img width="483" height="227" alt="image"
src="https://github.com/user-attachments/assets/4bf86eec-a9cc-4f63-84f0-4eb2bd535bab"
/>

## After

<img width="732" height="235" alt="image"
src="https://github.com/user-attachments/assets/101d2ea4-36e8-4e8f-990a-d72b33fa0ac2"
/>
2026-07-06 09:22:41 +00:00
James Brunton 11ba3814e5 Restructure Portal code to be inside Editor (#6857)
# Description of Changes
We don't have any strong reasons to keep the Portal as a separate Vite
app, and it needs access to so many things from the Editor that it no
longer makes sense to keep them separate. This PR moves the Portal code
to have direct access to the Editor code and gets rid of the shared
folder.
2026-07-03 13:20:02 +00:00
James Brunton c22ecc6c09 Add counts to sources page (#6819) 2026-07-01 11:42:35 +01:00
James Brunton 276eb8f2a7 Add pipelines page to portal (#6818)
# Description of Changes

Connect pipelines page to the backend. Note that this is really half an
implementation because the portal doesn't have access to the tools list
and their settings, but I can't fix that without re-architecture work,
which I'll do in another PR, then come back to finish this off in a new
PR.

<img width="786" height="579" alt="image"
src="https://github.com/user-attachments/assets/d3f06110-a35d-4d48-a2f9-1edb900c5c35"
/>

<img width="1232" height="519" alt="image"
src="https://github.com/user-attachments/assets/9f344648-ea45-498d-9e84-9558a3999838"
/>
2026-06-30 16:11:48 +00:00
James Brunton e44da5c410 Fix missing refresh token on desktop (#6838)
# Description of Changes
Fix #6801, along with fixing policies on desktop, which would attempt to
download policy outputs from the local backend instead of the server,
where they actually live. I've changed the policies logic to maintain
the same backend for the file retrieval as it used for the policy
running, so when we support running policies locally, it should still
work correctly.
2026-06-30 14:07:12 +00:00
James Brunton 82ec2acaba Make explicit signed and unsigned desktop CI jobs (#6840)
# Description of Changes
Makes it easier to skip signing on nightlies, which we don't need to do
since we're just warming the Rust cache.
2026-06-29 16:14:40 +00:00
James Brunton 6ff910f26c Expand any type linting in frontend (#6808)
# Description of Changes
Continued effort to expand linting scope to ban the `any` type in our
codebase. This PR pulls in a lot of subfolders into the linting scope,
because the excluded list was getting short enough that it was feasible
to move a layer down. I then fixed all the trivially fixable `any` type
violations in the subfolders, which just required local changes to the
one file. The aim of this PR is more to expand the scope to all the
folders we can that already avoid `any` types, rather than actually fix
violations.
2026-06-29 08:32:30 +00:00
James Brunton 013f145462 Upgrade to TS7 for local type-checking (#6815)
# Description of Changes
We can't convert to TS7 completely yet because it lacks the TS API, so
ESLint and some of our scripts don't work, but we can do [what the TS
team suggest and run TS6 and TS7
side-by-side](https://devblogs.microsoft.com/typescript/progress-on-typescript-7-december-2025/#compiler).
When we do that, we take the `task frontend:typecheck:all` job from ~76s
to ~13s, and everything else continues to work as it did before.

I've set it so that CI will still use TS6 for the time being and locally
we use TS7 out of an abundance of caution because CI time doesn't really
matter but local time does. I do think it was a bit pointless doing that
since the TS team claim the type checking performs identically, but we
might as well have it like that for now. If it happens to go badly
locally for any devs, they can use `CI=true task frontend:typecheck` to
revert to use TS6 trivially.
2026-06-26 15:05:07 +00:00
James Brunton eea9696bd4 Actually build the frontend for Playwright nightlies (#6817)
# Description of Changes
[Our nightlies have literally never passed
before](https://github.com/Stirling-Tools/Stirling-PDF/actions/workflows/nightly.yml).
As far as I can tell, that's because the frontend was never being built,
so the Playwright tests would just never start up.

I've forced a nightly run from this branch, and the Playwright tests
still fail, but for legitimate failures now. It's a separate job to
track down why they're actually failing, so I'm leaving that for
followup work.
2026-06-26 14:53:51 +00:00
James Brunton 3f7e898c69 Add sources service and frontend (#6774)
# Description of Changes
Redesign policies backend to treat sources a lot closer to how the
frontend imagined them working (they're persistent now and have an API).
Then connect the portal to the sources when mocks are off to allow for
source creation in the UI. It's not particularly useful to do that right
now because there's no policies UI, but I've tested manually that
sources set up in the UI are usable by policies created via the API.

I had to change the portal so that when mocks are off, it doesn't just
hard crash when attempting to connect to all the backend APIs that don't
exist yet. It'll still log the errors, but just continues on rendering
the UI now.

I also changed all the policies backend APIs to be gated behind a flag
instead of behind the SaaS profile. This is because we haven't yet got
the payment model sorted, but we're going to need this stuff running
self-hosted to be able to test it locally.
2026-06-26 13:21:53 +00:00
James Brunton def3cf79f6 More desktop CI optimisations (#6786)
# Description of Changes
- Change the nightly build to not sign any of the desktop builds, since
we just care about the compiled code. The restored code will still be
signed dependent on the OS in the PR builds.
- Change RPM Linux to use zstd for compression because the one it was
using runs really slowly, and the Jar is already compressed so it makes
basically no difference (arguably we shouldn't compress at all)
- ~Switch to consistently use Depot for Docker caching to stop filling
up the GHA cache and evicting the Rust cache~ Decided against switching
to Depot because we're probably doing another PR to remove Depot
altogether in the near future
2026-06-26 11:08:08 +00:00
James Brunton 41181c9da1 Redesign tool config types to avoid any typing (#6582)
# Description of Changes
Fixes one of the main causes of `any` typing left in tools, the way that
we register tool parameters in the registry. Currently, it just accepts
tool params via `any`, but instead we can explicitly change them to
`Record<string, unknown)`, so on the way back out they can more safely
be cast back to their correct type when known.

One consequence of this is that I had to redesign the way we
special-case the Convert tool, which previously was a different shape
than all the other param types. Now it's just got optional parameters on
it, which isn't quite as type-safe as before, but it does mean all tools
are a consistent shape now, which I think is worth the tradeoff.
2026-06-23 15:44:52 +00:00
James Brunton 101502cf4f Upgrade to TypeScript 6 (#6772)
# Description of Changes
TS6 introduced backwards-incompatible changes which affected us a little
bit. Other than that, I don't think it significantly changes things for
us, but we will need to deal with these breaking changes to be able to
upgrade to TS7 (the version written in Go, so dramatically faster), so
I'd rather do the work now before TS7 actually releases.

Main things I've done:
- Removed the use of `baseUrl` in the `tsconfig.json` files 
- Explicitly provide the `node` types where needed
- Explicitly reference the un-referenced but required Google API types
- Dropped the installation of `madge` which we weren't using and wasn't
directly compatible with TS6
- Updated the `i18next` packages for explicit TS6 compatibility
- Explicitly override `tsconfck` to force TS6 compatibility since we
can't upgrade it. We're only using that for `vite-tsconfig-paths` and it
all still seems to work fine, so I don't think this is an issue. I think
we can theoretically drop `vite-tsconfig-paths` when we upgrade to Vite
8 ([because it supports
`paths`](https://v8.vite.dev/guide/features#paths)), but that's a bigger
job than I want to do in this PR
2026-06-23 08:52:32 +00:00
James Brunton f2b65f4a77 Make pre-commit scripts more OS-agnostic (#6724)
# Description of Changes
Fix #6723
2026-06-23 08:42:18 +00:00
James Brunton 1816bad1ba Unified auth for portal and editor (#6725)
# Description of Changes
Refactor frontend auth to the shared folder and hook it up to both the
portal and editor so they share the same system. Also adds various tasks
to help run the portal, including `task dev:portal` to spawn the portal
with the backend, and `task dev:portal:proxy` to spawn the editor,
portal and backend, and a reverse proxy (at localhost:3000) to allow you
to use both at once to simulate how this will actually be deployed,
allowing you to check whether the seamless transition between the two
actually works.
2026-06-22 16:22:36 +00:00
James Brunton 9aee85d55e Wrap first login popup in a form so enter works to change password (#6769)
# Description of Changes

> [!note]
> GitHub absolutely mangles the diff unless you change to ignore
whitespace changes

This page has never had the Enter key bound to the Change Password
button:

<img width="673" height="745" alt="image"
src="https://github.com/user-attachments/assets/7a1b06f0-2945-4270-a795-f799ec556c12"
/>

This PR changes the modal to be properly wrapped in a form so key
commands work correctly on it.
2026-06-22 13:59:28 +00:00
James Brunton 72f8705460 Build Rust cache on nightlies (#6768)
# Description of Changes
Rust cache added in #6732 never fired because `main` builds don't
include building the desktop apps. We could build them on `main` builds,
but that's fairly expensive, so just build them on nightlies instead to
warm the cache for any desktop PRs the next day
2026-06-22 13:59:13 +00:00
James Brunton b9ea9064c7 Cache Rust build to improve Tauri build job times (#6732)
# Description of Changes
The Tauri jobs are very slow, especially the Linux ones, which can take
>1hr to build all the necessary code. A lot of that is because of the
actual Rust compilation, which isn't cached at all as far as I can tell.
This introduces a cache step for the Rust dependencies, so PRs will just
reuse the compiled Rust from the last build of main (if it's safe to do
so).
2026-06-19 15:27:14 +00:00
James Brunton 6a9876a067 Fix more any typing usage in the frontend (#6664)
# Description of Changes
Continued effort to remove the remaining uses of the `any` type from our
TS code. The vast majority of these uses that it cleans up was just
catching errors as `any`, which are pretty simple to fix. I couldn't
completely remove the `any` type usage from `core/tools` because there
were cascading issues from a couple of the files in there (most notably
Automate) but still, moving in the right direction.
2026-06-19 13:37:53 +00:00
James Brunton 3793a6df52 Fix bad frontend architecture (#6730)
# Description of Changes
#6727 introduced frontend code which goes against the architecture, so
this PR re-implements it in the architecture properly, along with
another bad Tauri check that I found in the source. I also updated the
`AGENTS.md` file to use Claude's "read this file" syntax to try and
force AI to actually read the file instead of just suggesting that it
does it.
2026-06-19 12:34:13 +00:00
James Brunton b57958531d Add message when running task with no arguments (#6731)
# Description of Changes
Add message describing the most common tasks when running `task` with no
arguments. I think this should help newcomers because `task --list` is
massive at this point and nobody's going to read through it all. Let me
know if you think any other commands should be in the default message.
2026-06-19 10:44:30 +00:00
James Brunton eb08e60d67 Redesign pre-commit commands to run through Task (#6670)
# Description of Changes
The `pre-commit` commands in this repo are inconsistent with the rest of
the dev workflow, as they are impossible to run through Task and they
can cause CI to fail with no way for a developer to run the `pre-commit`
scripts after they've failed. This PR adds `task pre-commit` (and `task
pre-commit:fix`) and then hooks up the existing `pre-commit` hooks and
CI to call the Task rule, so if developers are using pre-commit hooks
then they should still work, but they're also runnable without using
pre-commit at all.

I think it'd be worth reviewing what we're actually running at
pre-commit in the future because I'm not entirely convinced by all of
the scripts that we are running, but this should at least make what we
have properly enforced and usable by all devs.
2026-06-18 12:55:53 +00:00
James Brunton 0c503cc41d Fix all top-level dev tasks treating engine as enabled (#6705)
# Description of Changes
Currently, `task dev` explicitly calls the backend with
`AIENGINE_ENABLED=true` even though it isn't being spawned, so you just
get a dead FAB in the UI. This PR fixes it so that the engine will only
be enabled for tasks that will actually spawn the engine.

It also fixes a bug with the chat which makes it unusable locally. The
API path was not going through `apiClient` so for local dev you end up
with `//api/v1/...` which is not a valid path, so you get CORS errors
when trying to connect to the AI engine.
2026-06-18 10:08:50 +00:00
James Brunton 13af10a6d1 Redesign policy running (#6609)
# Description of Changes
Redesign policy running so the server is in charge of policy IDs and
running, to make it impossible to have the frontend miss the results.
This solves a minor bug that we currently have in policies, where if you
load a file and then refresh while the policy is running, you'll never
receive the outputted file.
2026-06-17 16:18:50 +00:00
James Brunton ef0deef4f2 Skip flaky Playwright test (#6698)
# Description of Changes
One of the Playwright tests is flaky, despite several attempts to fix it
before it made it into main. This disables the test for now so a
followup PR can try to fix it again.
2026-06-17 09:12:09 +00:00
James Brunton 10b4551449 Fix failing Playwright test in SaaS (#6688)
# Description of Changes
Fix Playwright failing test in SaaS (I think this is my third attempt
now so who knows if this will actually fix it for real this time, but
hopefully it does)
2026-06-16 15:56:30 +01:00
James Brunton 9a883be697 Cleanup of SaaS code (#6669)
# Description of Changes
De-AI comments and fix ridiculously indented code
2026-06-16 11:49:13 +01:00
James Brunton 42c1cce56d Fix Java formatting 2026-06-15 15:05:26 +01:00
fb6a118be9 Update SaaS to latest main (#6667)
# Description of Changes
> [!warning]
> **Do not** squash this on merge. It should be merged via a merge
commit

Fixes conflicts in `pgvector_store.py`. 

Also since codespell is failing, add comments to ignore the errors in
`sync_en_us_spelling.py`

---------

Co-authored-by: Ludy <Ludy87@users.noreply.github.com>
Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
2026-06-15 14:53:22 +01:00
James Brunton c55beacead Shut codespell up 2026-06-15 14:20:25 +01:00
James Brunton 04d68c650a Merge remote-tracking branch 'origin/main' into saas-update
# Conflicts:
#	engine/src/stirling/documents/pgvector_store.py
2026-06-15 14:10:21 +01:00
James Brunton 2a905c01c3 SaaS tidying (#6665)
# Description of Changes
* Remove complex port selection logic from `engine.yml`. It's
inconsistent with the frontend & backend task files, and caused issues
with Docker, which have been worked around but would be simpler to just
get rid of the problem altogether
* Fix Ruff formatting of Python script
* Remove payg tests which are failing and have drifted too far from the
implementation to save directly
2026-06-15 13:21:33 +01:00
James Brunton c1a637d764 Fix CI errors in SaaS (#6662)
# Description of Changes
Fix CI errors in #6578 to make SaaS branch ready for merge into main
2026-06-15 11:26:29 +01:00
James Brunton 511b92b321 De-AI the onboarding prose (#6641)
# Description of Changes
De-AI the onboarding prose.
2026-06-12 11:39:19 +01:00
James Brunton d363a1e957 Improve search logic (#6637)
# Description of Changes
Search has got significantly worse since #6581, where I added all the
missing tags for tools that should have been there for months. Turns out
that the fuzzy matching search logic has always been way too permissive
to match words with Levenshtein distances way too far away from the
target word, so long searches include way too much stuff. The new tags
just exposed that underlying logic issue. This PR makes the Levenshtein
logic much stricter, so it is still tolerant to minor typos in tool
names, but doesn't match completely inappropriate strings.
2026-06-12 10:34:11 +01:00
James Brunton ea102cdb93 Merge pull request #6636 from Stirling-Tools/SaaS-update
Update SaaS to latest main
2026-06-12 10:19:35 +01:00
James Brunton d995471a55 Merge remote-tracking branch 'origin/main' into SaaS-update
# Conflicts:
#	frontend/editor/src/proprietary/components/chat/ChatContext.tsx
#	frontend/editor/src/saas/components/shared/TrialStatusBanner.tsx
2026-06-12 09:58:40 +01:00
James Brunton 34ead60194 Kill off agents pane now that we have FAB (#6613)
# Description of Changes
Kill off agents pane now that we have the FAB. Also fixes a bug with the
FAB where it would sometimes fail to render the chat, and fixes a
duplicated entry in the Vite config which was throwing a warning
2026-06-11 17:28:05 +01:00
James Brunton d52c7ced7c Improvements to Stirling Engine to prepare for SaaS release (#6603)
# Description of Changes
- Use pool for postgres connections
- Add ability to require user ID to be set on API calls to the engine
- Add process-wide concurrency cap on AI access (in addition to existing
user caps)
- Allow number of workers (threads) to be specified for stirling engine
- Update env var names to reflect that the DB is not just for RAG
2026-06-11 16:31:35 +01:00
James Brunton 606964ee52 Fix Teams and MCP settings pages (#6605)
# Description of Changes
Remove the Pro guards from the Team settings page and also fix the
styling of the MCP settings screen (the code sections were black text on
black background in light mode)
2026-06-11 16:26:02 +01:00
James Brunton 68e031ac55 Policies tidying (#6587)
# Description of Changes
* Improve typing of API (breaking change but unreleased, frontend also
updated in this PR)
* Add ownership concept to policies
* De-AI the comments
* Update the `task dev:saas` rule to spawn the engine as well
2026-06-11 13:20:01 +01:00
James Brunton ebc28b0a14 Add team settings to SaaS (#6601)
# Description of Changes
Add team settings UI to SaaS, which is currently only available in
desktop. It'd be nice to refactor this so they're more shared, but
they're slightly different so needs to be done with some care. Leaving
for followup work.
2026-06-10 15:54:18 +01:00
James Brunton 3675db5907 Add new triggers, sources and sinks to policies (#6543)
# Description of Changes
Add new triggers:
- Schedule (fires every X amount of time)
- Folder watch (fires whenever the OS tells us a folder has a new file
in it; on Mac this is technically a 2s schedule but that's just how Java
implements it)

Add new sources:
- Folder (reads from this directory)

Add new sinks:
- Inline (stores in FileStorage)
- Folder (stores in specified directory)

Still want to do S3 buckets and web hooks and stuff, but they can come
in a future PR. I'm hoping this should make it sufficient to be able to
integrate with processing folders frontend etc. I've also changed it so
that policies can have multiple sources and triggers at once, which
seems like it might be useful.
2026-06-10 10:55:29 +00:00
James Brunton e0fc5061de Delete dead translations (#6581)
# Description of Changes
Adds all the missing translations that I could find (they're all dynamic
ones that the existing test can't detect are required) and adds a new
test to find as many unused translations as possible. The test has an
ignore list for translations that are used, but dynamically so the test
can't find them (most of the settings UI translations are built up
dynamically like that).

This PR is scoped to just include en-GB translation changes, since
that's the main supported language. We'll need to do a translation PR to
trim all the dead keys from the other languages, and add the missing
ones.
2026-06-10 09:49:00 +00:00
James Brunton 0e3cbb3cf2 Explicitly test for console warnings & errors (#6502)
# Description of Changes
Disallow warnings and errors from being thrown in the browser console
during tests unless explicitly expected in the test. Also adds a
Playwright test to prod around some main UI areas and checks that no
warnings/errors have been thrown.
2026-06-09 08:34:02 +00:00
James Brunton 002de06411 Fix desktop app not being able to load pdfium (#6575)
# Description of Changes
The changes in
[#6279](https://github.com/Stirling-Tools/Stirling-PDF/pull/6279) broke
the desktop app because the wasm URL handling didn't deal with
`tauri://` paths. Also I noticed that `task desktop:build:dev:mac`
failed locally because it was attempting to sign the app with
credentials that developers won't have (and shouldn't need), so I fixed
that too.
2026-06-08 16:21:56 +00:00
James Brunton 51478e5051 Policies backend (#6527)
# Description of Changes
Add a backend for running any multi-step PDF operations. This is
designed to be used for the upcoming Policies feature, along with
anything else that will require automated running of PDF operations,
like the Automate tool or Processing Folders.

The implementation is not complete. I've tried to get all the
infrastructure in there so that we can add in whichever triggers we need
in the future (like cron triggers or watching folders on disk) but
currently it just supports manual triggering of the policy.

The basis of this work was the operation running from the Stirling
Engine, which this PR removes in favour of this new system. The only
currently accessible frontend way to test this work is to ask the AI
chat to execute multiple operations on a PDF, but I've also extensively
tested with direct API calls to make sure that the policies work and
persist properly.
2026-06-08 10:50:55 +00:00
James Brunton e79f4a044f Make any typing linting opt-out instead of opt-in (#6542)
# Description of Changes
Reconfigure linting of `any` type to an opt-out instead of an opt-in
strategy now that we're close enough to everything supporting it. Also
slightly expands the scope of things included in the linting.
2026-06-05 14:11:26 +00:00
James Brunton 9ab404b2e6 Fix intermittently failing Playwright tests in main (#6541)
# Description of Changes
[#6474](https://github.com/Stirling-Tools/Stirling-PDF/pull/6474)
updated the IndexedDB schema number to v9, but a couple of Playwright
tests were explicitly creating a DB in v4 schema, which then caused
inconsistently failing tests because the DB upgrade process is
asynchronous and sometimes was too slow to upgrade, causing the test to
get into an invalid state.

Also fixes the screenshots directory exclusion since the frontend folder
was restructured.
2026-06-05 11:34:32 +00:00
James Brunton 7f3ca7ea70 Fix mockServiceWorker.js reformatting (#6526)
# Description of Changes
`mockServiceWorker.js` is a third-party managed file, which is included
in our `.prettierignore` file, and is rewritten to be in the module's
standard format whenever `msw` runs. At some point, it was reformatted
in our style, but shouldn't have been. This puts it back to `msw`'s
style, which should make it stop appearing in diffs.
2026-06-03 15:47:03 +00:00
James Brunton b705c5b84c Switch to use JPDFium v1.0.2, which signs the Mac binaries (#6521)
# Description of Changes
Currently, it's not possible to develop the backend on Mac without
manually signing the JPDFium binaries yourself since macOS will reject
running the unsigned binaries. [We've now updated JPDFium to sign the
Mac binaries in
v1.0.2](https://github.com/Stirling-Tools/JPDFium/releases/tag/v1.0.2),
so update to use that version.
2026-06-03 11:55:38 +00:00
James Brunton 1264f4cfed Set up document management for Stirling Engine (#6476)
# Description of Changes
Change Stirling Engine to support deleting documents automatically. This
happens both on user logout and after an amount of time specified by the
Java when ingesting a document (allowing for personal documents to have
short lifetimes but org documents to be left in the db with no expiry
date). Also sets up an [ACL
policy](https://en.wikipedia.org/wiki/Access-control_list) for the
documents so the database knows which users have access to which
documents. This is not fully implemented in the Java, so currently all
docs are treated as having a single owner, the uploader, but
theoretically when we need to support org storage, we shouldn't need to
change the db schema.
2026-06-03 11:52:11 +00:00
James Brunton 71633861d0 Make zoom key command behave the same regardless of mouse position (#6508)
# Description of Changes
Make zoom key command behave the same regardless of mouse position.
Previously only zoomed the editor if the mouse was over the editor.
2026-06-03 11:09:55 +00:00
James Brunton 2c01f41142 Update indexeddb to v9 to unify SaaS and OSS users (#6474)
# Description of Changes
The production SaaS is currently on v8 of IndexedDB due to various
schema changes for Smart Folders, which haven't made their way into OSS.
OSS is currently on v4 of IndexedDB, so if we release an OSS build to
the SaaS deployment, existing users will not be able to use it because
the DB version is 'too old'.

This PR updates the IDB version number to v9 so both OSS and SaaS users
will be able to upgrade to it. Theoretically both types of user should
be able to keep their IDB files without issue. SaaS previously actively
wiped the user's files in an old version (v6/v7) and users who haven't
used it since then will have their DBs wiped, but that'd happen anyway
if they use current SaaS so I don't think that matters.
2026-05-29 15:23:40 +00:00
James Brunton 4d5eeb103f Fix username display issues (#6471)
# Description of Changes
Main fixes:
- Fix the display of the username in the bottom left
- Now displays as "User" when not logged in on self-hosted (desktop) and
"Guest" on SaaS when logged in anonymously
- Now updates properly when the user logs in/out in SaaS, desktop and
self-hosted
- Fix incremental build issues in the desktop app that have been here
since the start (I hope at least - I think the issue is that the JLink
is built read-only and then on subsequent builds you get OS errors when
trying to override the JLink with the new version. There's no real need
for it to be read-only that I know of, so we might as well just make it
R/W and ship like that)
2026-05-29 14:35:47 +00:00
James Brunton 61ebe977d3 Auto-delete CI linting comments on success (#6465)
# Description of Changes
Set CI backend & engine comments to auto-delete once the CI has passed. 

Also redesign the engine CI to call `task engine:check` like it should
have been, and make it post a comment when the tool models need to be
updated.

Also makes the comment wording more consistent between the three
languages.
2026-05-29 10:13:12 +00:00
James Brunton 57af5b9dc2 Fix Tauri testing (#6462)
# Description of Changes
#6402 introduced a Rust test `refresh_token_fallback.rs`, but it wasn't
moved properly after the restructure of the `frontend/` folder in #6404.
This PR moves the file to the right place, and also hooks up Task and CI
rules for `cargo test` since nothing was actually running the test in
the first place.
2026-05-28 11:05:56 +00:00
James Brunton 44fbf8c587 Various bug fixes found while testing SaaS build (#6459)
# Description of Changes
Various fixes and improvements I made while testing the SaaS code:
- Changes the new `.env.saas` file to live in `app/` and match the
semantics of the other `.env` files
- Adds top-level `task dev:saas` command to spawn SaaS frontend &
backend
- Deletes dead SaaS code and improves some overriding logic
- Fixes refreshing issue when coming back to the tab
- Fix the Compare tool's selection logic
- Make Compare handle error cases properly
- Fixes the location of the "Dismiss All Errors" button (was rendering
on top of the top-bar with a transparent background previously so it
looked rubbish)
- Fixes file selection in PDF Editor
2026-05-28 11:05:30 +00:00
James Brunton d459ded168 Add cancel button to kill long-running AI tasks (#6351)
# Description of Changes
Adds a cancel button to the AI chat to allow the user to abort
long-running AI tasks. Just disconnects the SSE stream (all the backend
code already interrupts when it notices the stream is dead).
2026-05-28 09:25:23 +00:00
James Brunton beb99e273b Improve edit agent's knowledge of tools (#6356)
# Description of Changes
Give Edit Agent access to descriptions of the request from the Java API.
This opens the door to us better documenting our Java APIs to give the
stirling engine better knowledge of what the various tools are and how
to use them.

Also improves the tool selection sub-agent to get the tool parameters
and descriptions so it can more intelligently decide which operations
should be used to fulfil the user's request. Also provides it more
encouragement to string together multiple operations if necessary.
2026-05-14 18:30:39 +00:00
James Brunton 672e81d286 Add ability for Stirling engine to reason across large documents (#6314)
# Description of Changes
Adds storage in the database for full document content alongside the RAG
content (and changes the service to `DocumentService` instead of
`RagService`). Then adds a generic capability that should be usable by
any agent (currently just used by the Question Agent) which allows the
agent to pull out the full contents of the doc, chunks it into various
sections that will fit in the context window, and then processes them in
parallel to create an intermediate result, and then processes the
intermediate result into a final answer. It will re-chunk as many times
as necessary to get the content small enough for the actual answer to be
analysed (I've tested on PDFs ~3500 pages long, which is well above the
context limit and requires maybe 3 rounds of compression to get an
answer).

The new full doc analysis stuff is heavier than the RAG lookup so both
remain. The agents should use RAG for targeted info and the chunked
reasoner for info that requires reading the full doc.
2026-05-14 13:19:38 +00:00
James Brunton 8abe734f0b Fix main frontend validation (#6361)
# Description of Changes
#6312 reformatted `tauri.conf.json` via the Gradle script, which
reformats the entire file to not match the Prettier style. This PR
reformats the file back to Prettier format and changes the script to
update the version number without reformatting the entire file.

To be honest I'm not a huge fan of updating the version number with
regexes but it'd be a fool's errand to try and get Gradle to output JSON
in Prettier format, and this seems simpler than shelling out to run
Prettier over the file after the version string has been updated. Any
better ideas, let me know.
2026-05-14 12:31:45 +00:00
James Brunton c059e13423 Fix desktop app overscrolling inappropriately (#6350)
# Description of Changes
Fix #6348
2026-05-11 14:31:00 +00:00
James BruntonandEthanHealy01 575684ee4b Add edit text support to stirling engine (#6245)
# Description of Changes
Hooks up the (alpha) PDF Editor backend to the AI engine Edit Agent via
an intermediary API which is easier for the agent to call. It suffers
from all the same issues that the PDF Editor does in actually editing
the text, but should also benefit from any fixes to that.

It also adds protection against the underlying tools misbehaving by
hanging, and fixes a hanging bug in the PDF Editor.

---------

Co-authored-by: EthanHealy01 <80844253+EthanHealy01@users.noreply.github.com>
2026-05-11 09:57:41 +00:00
James Brunton 4ee47fdddd Fix Task commands in Powershell (#6330) 2026-05-08 10:47:36 +01:00
James Brunton 270af912d4 Fix frontend download links pointing to split Mac binaries (#6325) 2026-05-07 20:25:01 +01:00
James Brunton 8fb9fced35 Convert to building a universal Mac app (#6302) 2026-05-07 14:26:28 +01:00
James Brunton 95c39b4648 Fix missing desktop save indicator on files (#6310)
# Description of Changes
Save indicator stopped showing up after #6050, which fixed the missing
truncation on filenames, but accidentally bypassed the save indicator
component at the same time. This PR puts the component back in and makes
it support truncation so we can have both.

<img width="586" height="166" alt="image"
src="https://github.com/user-attachments/assets/529c3dcb-ee00-4a6d-ae53-ef8657204369"
/>
2026-05-05 11:19:11 +00:00