mirror of
https://github.com/Stirling-Tools/Stirling-PDF.git
synced 2026-09-02 21:03:34 +03:00
EthanHealy01
789be2d351
## Goal
Three related improvements to how policies and file state behave in the
editor: classification no longer blocks the user, policy enforcement
pipelines across a batch upload instead of waiting for the whole drop,
and file-state changes no longer re-render the entire UI.
## 1. Classification never blocks (and never versions)
Classification is metadata-only — it reads a document and records
labels; it never rewrites the file. Previously it ran like an
enforcement policy: it blocked viewing/editing behind the "Enforcing
policy…" overlay, forked a new versioned child (an `automate` entry in
version history), and could run before other policies — letting the user
in, then a later enforcement policy would fork a version and drop their
edits.
Now classification:
- **Never blocks.** A classification run never marks a file `enforcing`
(badge map + viewer overlay both skip it), so the file stays fully
viewable/editable while it runs.
- **No version bump, no history entry.** Its result is stamped onto the
file's existing stub in place (workspace + IndexedDB) — the labels just
appear as tags. It targets the document's *current leaf*, so an edit
made during the async run still gets the tags; a run that completes with
no outputs settles cleanly instead of pinning in-flight.
- **Always runs last** in an enforcement chain (regardless of configured
order, pinned at persist-time too), so every enforcement policy finishes
forking versions before the user is let in.
## 2. Pipeline policy enforcement across a batch upload
Dropping ~50 files enforced policies only *after the whole drop finished
scanning* — every file got the "Enforcing policy" overlay together, then
processing began. Root cause: the chunked `ADD_FILES` dispatches in
`addFiles` were never separated by an event-loop yield, so React batched
them into a single commit and the enforcement effect fired once over the
full list.
**Fix** (`core/contexts/file/fileActions.ts`): after each chunk, `await`
that chunk's IndexedDB writes, then yield a macrotask so React commits
the rows and runs the enforcement dispatch *before* the next chunk
scans. Files start enforcing as their rows land, overlapping with the
rest of the drop. Persistence is streamed per chunk (the policy auto-run
reads bytes from IndexedDB with no in-memory fallback).
**Second fix — bounded dispatch window**
(`proprietary/components/policies/usePolicyAutoRun.ts`): even with
streamed dispatch, the drop still *looked* serial — each dispatch POSTs
the file's bytes, and firing them all at once saturates the browser's
per-origin connection pool, so the status polls and output downloads of
already-running files queued behind the pending uploads; nothing visibly
progressed until the last upload drained. Dispatch is now gated behind a
small concurrency window (4), keeping connections free so early files
run, poll, and complete while later ones are still dispatching. The
first status poll also fires at 500ms (then the normal 2s cadence) so
fresh runs show real progress immediately. The batch test asserts the
window (dispatches overlap but never exceed 4).
## 3. Selector subscriptions for file state (no more whole-UI
re-renders)
`FileContext` published `{state, selectors}` through a plain React
context, so **every** consumer re-rendered on **every** state change —
one file's new version re-rendered the entire workspace.
**Phase 1 — infra** (`file/contexts.ts`, `file/fileHooks.ts`,
`FileContext.tsx`): the state context is replaced by a stable
subscription store (`FileStoreContext`); hooks are rebuilt on
`useSyncExternalStoreWithSelector` (the `use-sync-external-store` shim
react-redux uses — new direct dep, React 19 compatible). Each consumer
now re-renders only when its selected slice changes:
- `useStirlingFileStub(id)` → only that file's record
- `useAllFiles` → file-list changes only (immune to selection/UI churn)
- `useFileSelection`/`useSelectedFiles` → selection + the *selected*
files' records only
- `useFileUI` → its three UI scalars; `useFileContext` → files + pinned
slices
- `useFileState` keeps its whole-state contract for existing broad
consumers
A render-count test (`fileHooks.selector.test.tsx`) locks the bail-out
contract.
**Phase 2 — hot-path rows**: sidebar `FileItem` is memoized (with stable
empty-array props), so one file's change re-renders one row, not the
list. Active Files thumbnails were already memoized.
**Phase 3 — narrow the hottest consumers**: always-mounted whole-state
consumers migrated to slices — `Workbench`, `EmbedPdfViewer`, `Viewer`,
`NonPdfViewer`, `WorkbenchBar`, `ViewerContext`, `ViewerShareButton`,
`ZoomAPIBridge`, `ViewerAnnotationControls`, `ConvertSettings`,
`DismissAllErrorsButton`, `FileEditorThumbnail`,
`usePageEditorDropdownState`, `useSaveShortcut`, plus a new
non-subscribing `useFileSelectors()` for event-time reads
(`ReviewToolStep`, `useViewerReadAloud`, `useExitWarning`). Net effect:
selection/UI churn no longer re-renders the viewer/workbench, and a
version landing touches only components observing the files slice.
Broad readers (`FileSidebar`, `PageEditor`, `FileEditor`, `Redact`,
`FormFill`) deliberately stay on `useFileState` — they read most of the
state anyway.
**Hardening**: store notifications run in a layout effect (subscribers
re-render before paint — no stale frames), and outside production
`useFileSelectors()` wraps its selectors to `console.error` if one is
invoked during render (those reads don't subscribe, so render-time use
would silently go stale — not statically lintable, so it's guarded at
runtime; the full test suite passes under the guard).
## 4. Policy indicators: shared icons, non-blocking run chip, no pulse
- Badges and enforcement overlays now take their glyph from the shared
`policyCategoryIcon` map (the same source the processor's catalogue
uses) — label icon for classification, shield for security — instead of
a hardcoded shield everywhere.
- A non-blocking run (classification) shows a small accent-tinted pill
in the top-right of the Active Files card (category icon + loader) and
the normal spinning badge in the sidebar, via a new `background` badge
flag that nothing gates on. When the run finishes, the tagged files keep
a plain category badge.
- The post-run pulse/glow on sidebar badges is gone (with its `recent`
plumbing): spinner while running, static category icon when done.
## Verification
Full CI gate locally: `og:check`, `typecheck:all` (all variants),
`lint`, `format:check`, `build`, `test` (1366 — incl. the render-count
contract test, the classification-order/import unit tests, and the
61-file batch integration test driving the real dispatch → poll → import
→ chain effects), `storybook:build` — all green.
## Held for follow-up (not in this PR)
- **Reuse one PDFium engine across viewer file switches** (kills the
per-open "Loading PDF Engine" rebuild). Implemented on branch
`viewer/reuse-pdfium-engine`, but review found a confirmed leak
(orphaned PDFium handles when switching files mid-load); needs an
in-flight-load teardown before shipping.
Frontend
All frontend commands are run from the repository root using Task:
task frontend:dev— start Vite dev server (localhost:5173)task frontend:build— production buildtask frontend:test— run teststask frontend:test:watch— run tests in watch modetask frontend:lint— run ESLint + cycle detectiontask frontend:typecheck— run TypeScript type checkingtask frontend:check— run typecheck + lint + testtask frontend:install— install npm dependencies
For desktop app development, see the Tauri section below.
Layout
frontend/ is a workspace containing one or more apps. Today it holds the
PDF editor under frontend/editor/; new apps (the developer portal, etc.)
will sit alongside it as siblings. Shared tooling — package.json, node_modules,
.storybook/, ESLint, Prettier — lives at frontend/ so every app installs
once and lints with the same config.
Environment Variables
The editor's environment variables live in committed .env files at
frontend/editor/:
.env— used by all builds (core, proprietary, and as the base for desktop/SaaS).env.desktop— additional vars loaded in desktop (Tauri) mode.env.saas— additional vars loaded in SaaS mode
These files contain non-secret defaults and are checked into Git, so most dev work needs no further setup.
To override values locally (API keys, machine-specific settings), create an uncommitted sibling editor/.env.local / editor/.env.desktop.local / editor/.env.saas.local. Vite automatically layers these on top of the committed files.
Docker Setup
For Docker deployments and configuration, see the Docker README.
Tauri
All desktop tasks are available via Task. From the root of the repo:
Dev
task desktop:dev
This ensures the JLink runtime and backend JAR exist (skipping if already built), then starts Tauri in dev mode.
Build
task desktop:build
This does a full clean rebuild of the backend JAR and JLink runtime, then builds the Tauri app for production.
Platform-specific dev builds are also available:
task desktop:build:dev # No bundling
task desktop:build:dev:mac # macOS .app bundle
task desktop:build:dev:windows # Windows NSIS installer
task desktop:build:dev:linux # Linux AppImage
JLink Tasks
You can also run JLink steps individually:
task desktop:jlink # Build JAR + create JLink runtime
task desktop:jlink:jar # Build backend JAR only
task desktop:jlink:runtime # Create JLink custom JRE only
task desktop:jlink:clean # Remove JLink artifacts
Clean
task desktop:clean
Removes all desktop build artifacts including JLink runtime, bundled JARs, Cargo build, and dist/build directories.