mirror of
https://github.com/Stirling-Tools/Stirling-PDF.git
synced 2026-09-03 05:10:16 +03:00
saas-test
22
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 ConnorYohandJames Brunton
|
732ef18ae5 |
feat(account-link): redirect-based connect handshake for self-hosted linking (#7494)
Links a self-hosted instance to a SaaS team over an ordinary redirect, and leaves the admin's browser holding a Stirling session at the same time. ## The problem A self-hosted server needs a device credential bound to a SaaS team, and the admin's Supabase JWT must never reach the instance backend. Three things ruled out the obvious approaches: - **A customer hostname can never be in Supabase's redirect allow-list**, so the sign-in cannot happen on the instance's own origin. That is why SSO and sign-up did not work for linking at all. - **A device credential identifies a server, not a person.** Every attended portal read (Usage, Billing, Documents, Infrastructure) goes through `getPortalSaasToken()` and needs a *user* session, so a credential-only link left all of them asking for a second sign-in. - **The previous design relayed a JWT** from the browser into the instance, which is the thing we wanted to avoid. That path is deleted here. ## The solution Redirect and nonce, modelled on desktop's `authService.loginWithSelfHostedOAuth`: mint a nonce, hand the browser off, accept only a callback carrying that nonce back. Desktop has the OS route the reply; self-hosted has no OS hop, so our own approval page performs it. That is the point — the human half happens on an origin we control. ``` instance SaaS admin's browser | POST connect/request | | | (name, callback, nonce, | | | claim-secret hash) | | |-------------------------->| | | <- requestId + authorizeUrl | | | GET /link?request=... | | |<-------------------------------| | | sign in (SSO works here), | | | see ACCOUNT + ORIGIN, approve | | |------------------------------->| | | 302 callback#nonce+session | | POST connect/claim | | | (requestId, claim secret)| | |-------------------------->| | | <- device credential | | ``` Four properties carry the safety, and each is stated in the code because each is easy to lose in a refactor: - **The redirect target is never caller-supplied.** Validated once at creation, then read back from the stored row, so nothing in the approval page's URL can steer the token elsewhere. - **Approval and minting are separate.** Approval records the team and hands out nothing usable; the credential is minted only on claim, authenticated by a secret that never entered a browser. - **A re-authentication cannot move a server between teams.** The team is pinned at creation from the credential only that instance holds, so an approver from another team gets `WRONG_TEAM` instead of a rebind. - **The approver has to confirm what they are binding.** The page shows the address and the signed-in account, with a way to switch, and a checkbox naming the address gates the approve button. The name the server reports is deliberately not shown: the requester picks it on an unauthenticated endpoint, and its honest value is the hostname already in the address. The session rides the URL fragment, so it stays out of access logs and `Referer`, and is stripped before anything awaits. The claim is row-locked, so one approval mints once. A request lives 30 minutes; a settled one is not offered again, since approving it fails server-side. Signing in mid-flow no longer loses the request. The id is kept on the SaaS origin and resumed after any sign-in, which is what makes creating an account work: the confirmation email opens a new tab, where the `next` parameter is gone. Reading it does not consume it — the request may be open in two tabs — and only a recorded decision retires it. The result lands as a modal over the portal the admin started from, and the portal re-reads its link status so the page behind agrees with the modal. Plaintext `http://` callbacks are accepted rather than refused, because many self-hosted instances legitimately run plain HTTP on a private network; the address carries a warning icon explaining the risk, derived server-side so a requester cannot suppress it. Hard-refusing `http://` to a public IP literal is a reasonable follow-up; a bare hostname can't be classified without a DNS lookup, so the warning stays the general mechanism. ## Configuration Four surfaces. Placeholders below, not values. **SaaS backend** | Setting | Needed | Why | |---|---|---| | `stirling.billing.account-link.enabled` | Yes, `true` | The connect controller and service are `@ConditionalOnProperty` with no default, so without it the endpoints do not exist. | | `system.frontendUrl` | Only when the approval page is not on the API's own origin | Where the approver is sent. Must include the app's base path if it is served under one, or the redirect misses `/link`. | **SaaS frontend** | Setting | Needed | Why | |---|---|---| | `VITE_SUPABASE_URL`, `VITE_SUPABASE_PUBLISHABLE_DEFAULT_KEY` | Yes | Its own sign-in. Must be the project the SaaS backend validates tokens against. | | `RUN_SUBPATH` | Only if served under a subpath | Moves the approval page to `<base>/<subpath>/link`, so `system.frontendUrl` has to agree. | **Self-hosted backend** | Setting | Needed | Why | |---|---|---| | `stirling.billing.account-link.enabled` | Yes, `true` | Defaults to `false`. | | `stirling.billing.account-link.saas-base-url` | Yes | Origin of the SaaS API it links to. Not the SaaS frontend. | | `system.frontendUrl` | Optional | Externally reachable base URL for the callback. Otherwise derived from the request's `Origin`, which is right for ordinary deployments and wrong behind a rewriting proxy. | **Self-hosted frontend** | Setting | Needed | Why | |---|---|---| | `VITE_SUPABASE_URL`, `VITE_SUPABASE_PUBLISHABLE_DEFAULT_KEY` | Yes | Accepts the session handed over in the callback fragment. | | `VITE_SAAS_API_URL` | For Usage and Billing | Attended reads go to the SaaS API with the admin's token. Absent, those surfaces stay on the mock. | | `VITE_INCLUDE_PORTAL` | Production builds | Dev builds include the portal automatically; without it there is no link UI and no callback route. | Two things worth stating because neither fails loudly: - **Both frontends must use the URL *and* key of the same Supabase project**, and the same one the SaaS backend validates against. A key from one project with a URL from another is accepted by the browser and rejected by Supabase, which surfaces much later as "session expired" on Usage rather than as an error at hand-over. - **The Supabase redirect allow-list must contain the SaaS app's `/auth/callback`**, since a confirmation email returns through it. Entries are matched exactly. - **`system.frontendUrl` is the existing setting for this**, not a new one, so each side reads its own value and there is nothing extra to configure. It also gates share links, so on a stack with storage and sharing already on, setting it here turns those on too. The self-hosted side deliberately does **not** configure where the approval page lives — SaaS answers that in the connect-request reply, being the only party that knows. Also here, because testing this needs two stacks side by side: `linked:staging` / `linked:dev` (which derive `system.frontendUrl` and `RUN_SUBPATH` themselves), the missing `frontend:staging:saas`, and a per-mode vite `cacheDir` — two dev servers in different modes otherwise re-optimise over one shared dep cache. ## How to test Automated and green: `task frontend:check:all` plus both backend modules. `ConnectRequestServiceTest` covers callback validation, the per-IP cap, single-use approval, claim outcomes, expiry, `WRONG_TEAM` and reauth confirming without minting; `ConnectServiceTest` covers callback-resolution precedence including a foreign-origin callback being discarded; `ConnectControllerTest` covers the authorize URL, including the forwarded-header path and only the first hop being trusted; `ConnectCallback.test.tsx` covers the fragment being stripped synchronously and malformed fragments refused; `LinkAccountModal.test.tsx` covers link and reauth hitting different endpoints. Manual walkthrough: 1. `task linked:staging` — added here; brings up a SaaS stack and a self-hosted instance pointed at it, on discovered ports, and prints the four addresses. 2. Open the link-account modal in the self-hosted portal and continue. Expect the SaaS approval page at `/link?request=<id>`. 3. Sign in as a team leader, or create an account and confirm the email. Either way you should come back to the approval page. 4. Tick the acknowledgement and approve. Expect the fragment gone from the address bar immediately, a result modal over the portal, the portal showing linked without a reload, and attended reads (Usage, Billing) working without a second sign-in. 5. Repeat, approving as a member of a different team. Expect a refusal, not a rebind. ## Outstanding - #7415 to be reworked against this design once this lands. - **No SaaS-side UI to disconnect a server.** `GET /account-link/instances` and `POST /account-link/instances/{id}/revoke` are already team-scoped and leader-gated, and the portal has a panel that uses them, but `portal-saas/components/settings/accountLinkSettings.tsx` exports `null` on the reasoning that "SaaS has no account-link concept". That held when linking was a self-hosted admin managing their own instance; here a leader approves a server they may not administer, and has no way to withdraw it. The seam to fill is that one file. Expected to land with the CTA work in #7415. --------- Co-authored-by: James Brunton <jbrunton96@gmail.com> |
||
|
ConnorYoh
|
6bae9d516d |
chore(saas): one task per environment, and make the frontend follow it (#7483)
## The problem The `dev` profile hardcoded one project ref (`qacaivhsjtftfwtgjvva`) in five places: the ref, the Supabase URL, the publishable key, the datasource host and the meter endpoint. That made it both the shared environment everyone relies on *and* the only thing you could point the backend at. Testing an open SaaS PR meant hand-overriding all five via env just to reach that PR's Supabase preview branch, which is the only place the PR's migrations have actually been applied. Get it wrong and you see `relation "stirling_pdf.<new table>" does not exist` for a table the PR added, which is what happened on [#7414](https://github.com/Stirling-Tools/Stirling-PDF/pull/7414). ## One task per environment ```bash task dev:saas # backend + frontend + engine, against this PR's preview branch task staging:saas # backend + frontend + engine, against the shared v3 project task backend:dev:saas # backend only, preview branch task backend:staging:saas # backend only, v3 ``` | | how | vars | project | |---|---|---|---| | prod | `PROFILES=none` | `SAAS_DB_*` | the live one | | staging | `PROFILES=staging` | `SAAS_STAGING_*` | pinned to v3, always there | | dev | `PROFILES=dev` | `SAAS_DEV_*` | follows a SaaS PR's preview branch | `PROFILES` is still the underlying switch, so the old spelling keeps working. Production deliberately has no named task: reaching it should take a conscious `PROFILES=none`, not a tab-complete. **staging** is the old `dev` configuration, moved and kept pinned. The value of a shared environment is that it is still there tomorrow: reproduce a bug, paste a link to a colleague, share data. **dev** is parameterised by `SAAS_DEV_PROJECT_REF` and derives the Supabase URL, JWT issuer, JWKS, meter endpoint and (unless overridden) the database host from it. Switching which PR you are testing is one variable instead of five. With no ref set, `task backend:dev:saas` stops and says what to set rather than falling back. ## The frontend was the real gap `frontend/editor/.env` is committed and pins the **production** Supabase project, and nothing in the frontend knew about dev or staging. So `task dev:saas` gave you a backend on a preview branch and a login against prod, unless you happened to have hand-written `frontend/editor/.env.saas.local`. The dev tasks now read the backend's env files and derive `VITE_SUPABASE_URL` and `VITE_SUPABASE_PUBLISHABLE_DEFAULT_KEY` from the same project ref the backend resolved, so the two halves cannot point at different projects. Nothing to keep in sync by hand, no new vite mode, and `SAAS_ENV=prod` opts back out to the committed values. ## Where to put your local values Two files, both gitignored, neither ever committed: **`app/.env.saas.local`** is the only one you normally need. The tasks load it for the backend *and* the frontend. ```bash # staging: everything else is already defaulted, so this is all it takes SAAS_STAGING_DB_PASSWORD=... # dev: the preview branch of the PR you are testing, from its "Supabase Preview" check. # A branch has its OWN password and API keys; the parent project's will not authenticate. SAAS_DEV_PROJECT_REF=... SAAS_DEV_DB_PASSWORD=... SAAS_DEV_PUBLISHABLE_KEY=... # prod, if you ever need it SAAS_DB_PROJECT_REF=... SAAS_DB_URL=... SAAS_DB_PASSWORD=... SUPABASE_EDGE_FUNCTION_SECRET=... ``` **`frontend/editor/.env.saas.local`** is no longer needed for choosing a Supabase project, and is best left empty or deleted. If you have one from before this PR, note that the task-supplied values now win, which is the point: the frontend follows the backend. **A blank is not the same as absent.** A dotenv line with an empty value still *sets* the variable, and Spring's `${VAR:default}` only falls back when a variable is absent. So `.env.saas` lists what you must set as blanks, and leaves out the two `*_DB_URL` overrides, which have real defaults to fall back to. This is not theoretical, see below. Committed `app/.env.saas` holds non-secret defaults only. Real secrets are passwords, the edge-function secret and service-role keys. Project refs and publishable keys are neither: a ref is the public `<ref>.supabase.co` subdomain and a publishable key ships in the browser bundle by design, which is why `frontend/editor/.env` has always carried prod's. ## Three bugs found while building the tasks All three were in this PR's own earlier commits, and all three were caught by actually booting things rather than by reading the config. **staging could not boot at all.** A blank `SAAS_STAGING_DB_URL=` in `.env.saas` set the variable to empty, so `${SAAS_STAGING_DB_URL:jdbc:...}` resolved to `""` and startup failed with `spring.datasource.url is required when the saas profile is active`. The file already carried a comment warning about exactly this; it had only been applied to the dev block. The original verification for this PR was "placeholders resolve" and "the task parses", neither of which boots anything. **The dev to staging fallback ran `ddl-auto=update` against shared v3.** The dev profile sets `update`, which is right for a disposable preview branch, and separately fell back to staging's project ref. Together that meant Hibernate was free to reconcile tables that RLS policies depend on. `application-staging.properties` pins `none`, but that only applies when the staging profile is the active one, which it was not on the fallback path. There is no fallback now: with no ref the task stops before gradle, and the frontend fails the same way, both naming the variable. **`PROFILES=` never selected production.** Go template `default` treats `""` as absent, so it silently resolved back to `dev`. It is `PROFILES=none` now. ## Two choices worth reviewing **Staging keeps its committed project ref**, now as a `${SAAS_STAGING_PROJECT_REF:...}` default in one place, with the URL, database host and meter endpoint all derived from it. So staging still works with zero setup, and repointing it is one variable. Nothing in CI referenced the ref or the profile. Its publishable key default carries no inline `gitleaks:allow`: a trailing comment in a `.properties` file is part of the value, so the pragma ended up inside the key. It is in `.gitleaksignore` instead. **`SAAS_DEV_DB_URL` still overrides the whole URL**, so a branch needing the pooler host rather than the direct one is reachable without touching committed config. ## Verification - `task backend:staging:saas` boots against v3 and serves `200`. It could not boot before this commit. - `task backend:dev:saas` with no ref stops before gradle naming the variable, and `PROFILES=none` still reaches production. `task frontend:dev:saas` fails the same way; `SAAS_ENV=staging` still resolves with no local config. - Frontend routing picks the SaaS runner for dev/staging and the plain runner for prod; the derivation returns the right URL and key for each. - Vite's `process.env` precedence and Task's dotenv/env semantics were measured, not assumed. That is how one trap surfaced: Task sets an `env:` key even when its value resolves to empty, and Vite treats an empty `process.env` `VITE_*` as authoritative over a committed `.env`. Putting the Supabase vars on the shared `dev:_run` would have blanked Supabase config for the core, proprietary and desktop dev servers, so the SaaS path has its own runner. - `:saas:spotlessApply` and `:saas:compileJava` green. `DevProfileProjectNotice` becomes `SaasProjectNotice` and covers both profiles, stating the project ref and `ddl-auto` at startup so which environment you are on is never a guess. No behaviour change for prod: the `saas` profile is untouched. |
||
|
Anthony Stirling
|
0be10b2dff |
Cucumber concurrency validation plus fix (#7379)
# Description of Changes cucumber tests to run multiple threads of commands at same time --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. |
||
|
James BruntonandAnthony Stirling
|
cd199c8659 |
Define tool inputs & outputs in a structured way (#7204)
# Description of Changes Change tool APIs to use structured definitions for input/output/type info because we need that info to be able to validate whether policies can actually successfully work based on whether one tool accepts the output of another. There were various bugs in the previous string definitions because of either misspellings or just incorrect definitions, so I've gone through and fixed all that I can find. <img width="729" height="271" alt="image" src="https://github.com/user-attachments/assets/08357e96-6fbb-4b9c-ba4d-8995420c7b86" /> <img width="749" height="264" alt="image" src="https://github.com/user-attachments/assets/76f46284-1866-4b64-b1ed-2480e01866e9" /> <img width="402" height="636" alt="image" src="https://github.com/user-attachments/assets/8f7a36ca-2845-4f14-a2df-ec9c772e66f6" /> <img width="393" height="317" alt="image" src="https://github.com/user-attachments/assets/46d8b891-9820-4ce3-8109-a8b782277037" /> --------- Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com> |
||
|
James Brunton
|
ed58d90ab8 |
Remove policies feature flag (#7031)
# Description of Changes Removes the feature flags for enabling policies on both the backend and frontend. We shouldn't be releasing another self-hosted release that doesn't include policies, so it makes sense to do this now. Builds that don't have the Processor will just not run policies because they won't have any. Beyond that, the API should always be available, but checks whether the user actually has the entitlements to run policies (whether they have credits/a payment method available) |
||
|
James Brunton
|
b4f7b1d8a9 |
Add bidirectional API types to frontend (#6867)
# Description of Changes Fix https://github.com/Stirling-Tools/Stirling-PDF-SaaS/issues/281. Add generated backend API mappings to the frontend code, and the logic to convert from a backend API to frontend parameters objects. Previously, it was impossible to tell if changing the backend API would require a change to the frontend to support it because the frontend had no static type information about the backend API. This PR adds autogenerated tool API types to the frontend (in `toolApiTypes.ts`) and adds explicit typed mappings between the frontend parameter types and the backend API types, so theoretically the type checker should be able to catch issues when changing one puts us in an invalid state with the other. During development, it pointed out several inconsistencies that we have between the frontend and backend types, some of which were genuine bugs, and others were only happening to work because the backend is more permissive than its API claims to be. This also unlocks the ability for us to render the frontend settings on saved backend API structures, which we've previously had to avoid doing because we had no reverse mapping. |
||
|
James Brunton
|
11ba3814e5 |
Restructure Portal code to be inside Editor (#6857)
# Description of Changes We don't have any strong reasons to keep the Portal as a separate Vite app, and it needs access to so many things from the Editor that it no longer makes sense to keep them separate. This PR moves the Portal code to have direct access to the Editor code and gets rid of the shared folder. |
||
|
Reece Browne
|
467f3a86c4 | Portal policies (#6852) | ||
|
James Brunton
|
3f7e898c69 |
Add sources service and frontend (#6774)
# Description of Changes Redesign policies backend to treat sources a lot closer to how the frontend imagined them working (they're persistent now and have an API). Then connect the portal to the sources when mocks are off to allow for source creation in the UI. It's not particularly useful to do that right now because there's no policies UI, but I've tested manually that sources set up in the UI are usable by policies created via the API. I had to change the portal so that when mocks are off, it doesn't just hard crash when attempting to connect to all the backend APIs that don't exist yet. It'll still log the errors, but just continues on rendering the UI now. I also changed all the policies backend APIs to be gated behind a flag instead of behind the SaaS profile. This is because we haven't yet got the payment model sorted, but we're going to need this stuff running self-hosted to be able to test it locally. |
||
|
James Brunton
|
1816bad1ba |
Unified auth for portal and editor (#6725)
# Description of Changes Refactor frontend auth to the shared folder and hook it up to both the portal and editor so they share the same system. Also adds various tasks to help run the portal, including `task dev:portal` to spawn the portal with the backend, and `task dev:portal:proxy` to spawn the editor, portal and backend, and a reverse proxy (at localhost:3000) to allow you to use both at once to simulate how this will actually be deployed, allowing you to check whether the seamless transition between the two actually works. |
||
|
James Brunton
|
b57958531d |
Add message when running task with no arguments (#6731)
# Description of Changes Add message describing the most common tasks when running `task` with no arguments. I think this should help newcomers because `task --list` is massive at this point and nobody's going to read through it all. Let me know if you think any other commands should be in the default message. |
||
|
James Brunton
|
eb08e60d67 |
Redesign pre-commit commands to run through Task (#6670)
# Description of Changes The `pre-commit` commands in this repo are inconsistent with the rest of the dev workflow, as they are impossible to run through Task and they can cause CI to fail with no way for a developer to run the `pre-commit` scripts after they've failed. This PR adds `task pre-commit` (and `task pre-commit:fix`) and then hooks up the existing `pre-commit` hooks and CI to call the Task rule, so if developers are using pre-commit hooks then they should still work, but they're also runnable without using pre-commit at all. I think it'd be worth reviewing what we're actually running at pre-commit in the future because I'm not entirely convinced by all of the scripts that we are running, but this should at least make what we have properly enforced and usable by all devs. |
||
|
James Brunton
|
0c503cc41d |
Fix all top-level dev tasks treating engine as enabled (#6705)
# Description of Changes Currently, `task dev` explicitly calls the backend with `AIENGINE_ENABLED=true` even though it isn't being spawned, so you just get a dead FAB in the UI. This PR fixes it so that the engine will only be enabled for tasks that will actually spawn the engine. It also fixes a bug with the chat which makes it unusable locally. The API path was not going through `apiClient` so for local dev you end up with `//api/v1/...` which is not a valid path, so you get CORS errors when trying to connect to the AI engine. |
||
|
James Brunton
|
13af10a6d1 |
Redesign policy running (#6609)
# Description of Changes Redesign policy running so the server is in charge of policy IDs and running, to make it impossible to have the frontend miss the results. This solves a minor bug that we currently have in policies, where if you load a file and then refresh while the policy is running, you'll never receive the outputted file. |
||
|
James Brunton
|
68e031ac55 |
Policies tidying (#6587)
# Description of Changes * Improve typing of API (breaking change but unreleased, frontend also updated in this PR) * Add ownership concept to policies * De-AI the comments * Update the `task dev:saas` rule to spawn the engine as well |
||
|
EthanHealy01
|
763595a5a3 |
feat: add Agents UI to proprietary right sidebar (#6454)
Update UI to include agents Run `task dev:all` to test |
||
|
James Brunton
|
44fbf8c587 |
Various bug fixes found while testing SaaS build (#6459)
# Description of Changes Various fixes and improvements I made while testing the SaaS code: - Changes the new `.env.saas` file to live in `app/` and match the semantics of the other `.env` files - Adds top-level `task dev:saas` command to spawn SaaS frontend & backend - Deletes dead SaaS code and improves some overriding logic - Fixes refreshing issue when coming back to the tab - Fix the Compare tool's selection logic - Make Compare handle error cases properly - Fixes the location of the "Dismiss All Errors" button (was rendering on top of the top-bar with a transparent background previously so it looked rubbish) - Fixes file selection in PDF Editor |
||
|
ConnorYoh
|
b146d9994d |
fix(task): make task dev / task dev:all work on Windows (#6392)
## Summary
`#6145` (port picker) and `#6244` (gradle unification) combined to break
`task dev` / `task dev:all` on Windows. Two independent regressions,
both addressed here.
### 1. `find-free-port.ps1` panic (`#6145`)
```
$ task --dry dev
The argument 'scriptsfind-free-port.ps1' to the -File parameter does not exist.
panic: ended up with a non-nil exitStatus.err but a zero exitStatus.code
```
- **Backslash stripped.** `Taskfile.yml` had
`scripts\find-free-port.ps1`. go-task pipes the `sh:` block through
mvdan/sh, which treats `\` as a POSIX escape and silently drops it,
leaving `scriptsfind-free-port.ps1`. PowerShell can't find the file,
exits non-zero, mvdan/sh panics on the inconsistent exit status.
Switched to a forward slash.
- **`-Preferred 8080,5173` is brittle.** Relies on PowerShell parsing
the comma-list into `[int[]]`. Dropped the named flag; switched the
script's `param` to `[Parameter(ValueFromRemainingArguments =
$true)][int[]]$Preferred`; pass each port as its own positional token
(matching how the bash variant is called).
### 2. `bash gradlew` can't find Java on Windows (`#6244`)
```
[backend:dev] ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
```
`#6244` unified all gradle invocations under `bash gradlew` on the
assumption that Git-Bash inherits Windows-side env. It doesn't (and
neither does WSL bash, which can also shadow `bash` on a developer's
PATH). The standard Adoptium/Temurin installer sets `JAVA_HOME` in the
Windows env only, so `bash gradlew` fails before Spring Boot even loads.
Restored the per-platform branch for every backend task: `cmd /c
".\gradlew.bat ..."` on Windows, `./gradlew ...` on Linux/macOS. Two
Windows-specific gotchas to be aware of for future edits:
- mvdan/sh strips `\` outside of double quotes, so `cmd /c
.\gradlew.bat` ends up as `.gradlew.bat`. The entire payload must be
wrapped in double quotes (`cmd /c "..."`).
- Modern Windows excludes cwd from cmd.exe's search path, so the leading
`.\` is required — bare `cmd /c gradlew.bat` errors with "is not
recognized" even from the repo root.
## Test plan
- [x] `task --dry dev` / `task --dry dev:all` on Windows resolve ports
cleanly and dispatch the inner tasks
- [x] `powershell -NoProfile -File scripts/find-free-port.ps1 8080 5173
5001` prints three ports
- [x] `task backend:dev PORT=8081` on Windows starts Spring Boot in ~9s;
`/api/v1/info/status` returns `{"status":"UP"}`
- [x] `task dev:all` on Windows brings up all three services healthy:
backend `/api/v1/info/status` 200, engine `/health` 200, frontend `/`
200
- [ ] Linux / macOS path unaffected (only the Windows branches changed
in behaviour)
|
||
|
Anthony StirlingandJames Brunton
|
f60a075443 |
Add Playwright/bootRun/test.sh tasks (#6244)
## Description Consolidates Playwright running under cohesive Task namespaces, isolates Playwright state from the developer's local working tree, and swaps CI's frontend webserver from `vite` dev to `vite preview` against a pre-built `dist/`. ### `e2e:*` namespace Renames `.taskfiles/testing.yml` to `.taskfiles/e2e.yml` and consolidates everything Playwright-related under one `e2e:` namespace: - `e2e:stubbed` / `e2e:live` / `e2e:enterprise` / `e2e:cross-browser`: project-specific runners - `e2e:check` (no-Docker subset) and `e2e:check:all` (full) - `e2e:oauth:up` / `:down`, `e2e:saml:up` / `:down`: symmetric lifecycle for the keycloak compose stacks - `e2e:install`: Playwright browser install - `docker:test`: full Docker integration suite The redundant `frontend:test:e2e:*` project shortcuts are removed. CI workflows (`e2e-stubbed.yml`, `e2e-live.yml`, `build-enterprise.yml`, `nightly.yml`) are updated to call the new task names. ### Isolated Playwright state New `STIRLING_BASE_PATH` (and `-Dstirling.base-path=`) override in `InstallationPathConfig` redirects the entire state tree (configs, backups, customFiles, pipeline, logs) at startup. `task e2e:live` points it at `.test-state/playwright/` (purged on every invocation) so the suite never touches the developer's local DB, settings.yml or backups. `task e2e:live` auto-spawns gradle, waits for `/api/v1/info/status` to come up, runs Playwright, then tears down the whole backend process tree. ### CI runs Playwright against `vite preview` Builds the frontend up-front with `VITE_BUILD_FOR_PREVIEW=1` (forces absolute base so deep SPA routes resolve `/assets/...`) and the playwright `webServer` now uses `vite preview --port 5173 --strictPort` in CI. Avoids the per-page on-demand transform cost that was blowing the 30s navigation timeout under `--workers=3` on `all-tool-pages-load.spec.ts`. Local dev keeps `vite` dev for HMR. ### OAuth/SAML compose helpers `start-oauth-test.sh` and `start-saml-test.sh` gain a `--license-key <KEY>` (`-k`) flag so CI and scripted runs can skip the interactive license prompt. `start-oauth-test.sh` also moves from `for arg in "$@"` to a `while`-with-`shift` arg loop to support multi-arg flags consistently with the SAML script. ### Backend gradlew unification Drops the per-platform `cmd /c gradlew.bat` branches from `backend.yml` and routes every gradle invocation through `bash gradlew`. Works uniformly on Linux/macOS and Windows-with-Git-Bash. ### Compare.tsx flake fix (re-land of [#6316](https://github.com/Stirling-Tools/Stirling-PDF/pull/6316)) Piggybacks Anthony's never-merged fix from #6316. Without it, `e2e:stubbed` continues to flake under `--workers=3` on `compare.spec.ts`'s second-upload case via a React "Maximum update depth exceeded" infinite loop in the Compare auto-fill effect. CI traces from recent failed runs match exactly; 10 local runs of `compare.spec.ts` with `CI=1 --workers=3` pass cleanly with the fix applied. --------- Co-authored-by: James Brunton <jbrunton96@gmail.com> |
||
|
James Brunton
|
49f24fa7a9 | Have Task choose free ports for dev servers (#6145) | ||
|
|
e5767ed58b |
Change AI engine to execute tools in Java instead of on frontend (#6116)
# Description of Changes Redesign AI engine so that it autogenerates the `tool_models.py` file from the OpenAPI spec so the Python has access to the Java API parameters and the full list of Java tools that it can run. CI ensures that whenever someone modifies a tool endpoint that the AI enigne tool models get updated as well (the dev gets told to run `task engine:tool-models`). There's loads of advantages to having the Java be the one that actually executes the tools, rather than the frontend as it was previously set up to theoretically use: - The AI gets much better descriptions of the params from the API docs - It'll be usable headless in the future so a Java daemon could run to execute ops on files in a folder without the need for the UI to run - The Java already has all the logic it needs to execute the tools - We don't need to parse the TypeScript to find the API (which is hard because the TS wasn't designed to be computer-read to extract the API) I've also hooked up the prototype frontend to ensure it's working properly, and have built it in a way that all the tool names can be translated properly, which was always an issue with previous prototypes of this. --------- Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com> Co-authored-by: EthanHealy01 <80844253+EthanHealy01@users.noreply.github.com> |
||
|
ConnorYohandJames Brunton
|
702f4e5c2c |
Add Taskfile for unified dev workflow across all components (#6080)
## Add Taskfile for unified dev workflow ### Summary - Introduces [Taskfile](https://taskfile.dev/) as the single CLI entry point for all development workflows across backend, frontend, engine, Docker, and desktop - ~80 tasks organized into 6 namespaces: `backend:`, `frontend:`, `engine:`, `docker:`, `desktop:`, plus root-level composites - All CI workflows migrated to use Task - Deletes `engine/Makefile` and `scripts/build-tauri-jlink.{sh,bat}` — replaced by Task equivalents - Removes redundant npm scripts (`dev`, `build`, `prep`, `lint`, `test`, `typecheck:all`) from `package.json` - Smart dependency caching: `sources`/`status`/`generates` fingerprinting, CI-aware `npm ci` vs `npm install`, `run: once` for parallel dep deduplication ### What this does NOT do - Does not replace Gradle, npm, or Docker — Taskfile is a thin orchestration wrapper - Does not change application code or behavior ### Install ``` npm install -g @go-task/cli # or: brew install go-task, winget install Task.Task ``` ### Quick start ``` task --list # discover all tasks task install # install all deps task dev # start backend + frontend task dev:all # also start AI engine task test # run all tests task check # quick quality gate (local dev) task check:all # full CI quality gate ``` ### Test plan - [ ] Install `task` CLI and run `task --list` — verify all tasks display - [ ] Run `task install` — verify frontend + engine deps install - [ ] Run `task dev` — verify backend + frontend start, Ctrl+C exits cleanly - [ ] Run `task frontend:check` — verify typecheck + lint + test pass - [ ] Run `task desktop:dev` — verify jlink builds are cached on second run - [ ] Verify CI passes on all workflows --------- Co-authored-by: James Brunton <jbrunton96@gmail.com> |