Compare commits

...
Author SHA1 Message Date
Anthony Stirling 02b9923ef1 Enforce sharing egress policies on document delivery on the editor pipeline base 2026-08-31 22:20:49 +01:00
James Brunton 763cb2dcca Minor bug fix 2026-08-28 15:15:48 +01:00
James Brunton ee0ae22c3c Fix comments 2026-08-28 13:49:48 +01:00
James Brunton ae4b869d3f Fix comments 2026-08-28 13:40:01 +01:00
James Brunton c4a72685c9 Redesign policy running to be generic with local and server passes 2026-08-28 13:40:01 +01:00
James Brunton 0fd3da7cd2 Fix classification policy activation 2026-08-28 13:40:01 +01:00
James Brunton df94eab93a Filter fake source out 2026-08-28 13:40:01 +01:00
James Brunton 6dfd894bb0 Fix race 2026-08-28 13:40:01 +01:00
James Brunton ff79a86e45 Fix test 2026-08-28 13:40:01 +01:00
James Brunton 8b0ae08b0b Finish removing legacy virtual source usages 2026-08-28 13:40:01 +01:00
James Brunton c04e59dae3 Formatting 2026-08-28 13:40:00 +01:00
James Brunton 33188cc0f4 Remove pointless casts 2026-08-28 13:40:00 +01:00
James Brunton 5713e38993 Update tests 2026-08-28 13:40:00 +01:00
James Brunton 6a42e0be49 Fix comment 2026-08-28 13:40:00 +01:00
James Brunton 1f85515154 Fix local polling bug 2026-08-28 13:40:00 +01:00
James Brunton 12b05a59c1 Change policy running to be generic 2026-08-28 13:40:00 +01:00
Anthony Stirling 89428d6b81 Seed the classification policy with the editor listed as a source 2026-08-28 13:40:00 +01:00
Anthony Stirling 595b9cbc85 Carry editor participation through policy writes instead of dropping it 2026-08-28 13:40:00 +01:00
Anthony Stirling b7935f9bd9 Lift editor participation from legacy policy options on read 2026-08-28 13:40:00 +01:00
Anthony Stirling 37c8621426 Move editor participation onto the policy and address review feedback 2026-08-28 13:40:00 +01:00
Anthony Stirling 64eb44e627 Cover editor pipeline auto-run in the stubbed browser suite 2026-08-28 13:40:00 +01:00
Anthony Stirling c241c1c28d Hold export-time enforcement to the same runsOnEditor answer as upload 2026-08-28 13:40:00 +01:00
Anthony Stirling 439fcb8756 Let a pipeline run on the editor, on upload or export 2026-08-28 13:40:00 +01:00
97 changed files with 5506 additions and 1199 deletions
+58
View File
@@ -15,6 +15,7 @@ The File Sharing feature enables users to store files server-side and share them
- Storage quotas (per-user and total)
- Pluggable storage backend (local filesystem or database BLOB)
- Integration with the Shared Signing workflow
- Policy enforcement at egress (who may receive a document, on what terms, and what the copy that leaves looks like)
## Architecture
@@ -29,6 +30,8 @@ The File Sharing feature enables users to store files server-side and share them
**`file_shares`**
- One record per sharing relationship
- `egress_channel` - the share channel this row was granted through, so a delivery is judged the way the grant was (an email share mints a link, and that link is still an email share)
- `egress_file_id` / `egress_fingerprint` — cache of the processed copy a Sharing policy produced for this share (see [Sharing Policies](#sharing-policies-egress-enforcement))
- Two share types, distinguished by which fields are set:
- **User share**: `shared_with_user_id` is set, `share_token` is null
- **Link share**: `share_token` is set (UUID), `shared_with_user_id` is null
@@ -296,6 +299,61 @@ GET /api/v1/storage/files/{fileId}/shares/links/{token}/accesses
Returns per-user access history (username, VIEW/DOWNLOAD, timestamp), sorted descending by time.
## Sharing Policies (egress enforcement)
A **Sharing** policy governs how documents leave the workspace. It is an ordinary stored policy (`/api/v1/policies`) authored under the `sharing` category, so it is created and edited in the portal's Policies page like any other - but it has no input source and nothing sweeps or polls for it. The sharing endpoints evaluate it inline, on the request that grants or delivers a share.
The governing policies are always the **file owner's team's**, never the accessor's.
### Where it fires
| Moment | Endpoint | What the policy can do |
|--------|----------|------------------------|
| Share with a user | `POST /files/{id}/shares/users` | refuse the share; cap the access role |
| Create a share link | `POST /files/{id}/shares/links` | refuse; cap the role; cap the link's lifetime |
| Email share | `POST /files/{id}/shares/users` (email address) | as above, judged as its own channel |
| Recipient opens it | `GET /share-links/{token}`, `GET /files/{id}/download` | force view-only; serve a processed copy |
The channel is stamped on the share row when it is granted (`file_shares.egress_channel`), so the link an email share mints is still evaluated as `emailShare` when the recipient opens it. Without that stamp a policy narrowed to the email channel would gate the grant and then let the delivery through untouched.
Typing a **registered user's** email address is still a user share: the row that names them is stamped `userShare`, and only the link that gets mailed is an `emailShare`. Both channels are evaluated before anything is written, each against the artefact it governs. Usernames in this product are commonly email addresses, so treating the row as an email share would quietly stop a `userShare` policy from biting on the ordinary path.
The owner downloading their own file is not egress and is never gated.
### Settings
| Setting | Effect |
|---------|--------|
| Default access | Ceiling on the role any new share may grant (`restricted``VIEWER`). Only tightens; it never promotes a weaker request. |
| External recipients | `allow` / `restrict` / `block`. Restrict grants read-only, view-only access on a one-day link. |
| Internal domains | Email domains counted as inside the org. Empty falls back to the file owner's own domain. |
| Share link expiry | Ceiling on link lifetime. Can only shorten the configured `linkExpirationDays`, never extend it. |
| Downloads | `viewOnly` always serves the document inline, never as an attachment, and rasterises the copy that leaves. |
Narrowing: **Runs on** limits the policy to particular share channels (`userShare`, `shareLink`, `emailShare`); empty means every channel.
Narrowing by document type is deliberately not offered. The `scopeTypes` field still round-trips in the stored format, but nothing sets or reads it: classification labels are written into the PDF only by the AI classify endpoint, and only into the editor's copy, so a file uploaded straight to storage carries none. A document-type rule would silently match nothing — worse than not having the control. It can return once stored files carry classification.
Several sharing policies compose to the most restrictive outcome: lowest role, shortest link life, view-only if any says so, blocked if any blocks.
### The processed copy
If the policy has any enabled tools (watermark, redact, strip active content), the recipient gets a **processed copy**; the stored original is untouched. It is derived on first delivery and cached against the share (`file_shares.egress_file_id`), keyed by a fingerprint over the transforming policies, their steps, and the source document's version — so editing the policy or replacing the document invalidates it automatically. The copy lives in the shared job file store, so it inherits that store's retention sweep and does not count against the owner's quota.
Three behaviours worth knowing:
- **Fails closed.** If the tool chain errors or does not finish inside 120s, the delivery is refused rather than falling back to the unprocessed original.
- **No signed-URL shortcut.** When a policy applies view-only or a transform, delivery always streams through the application. A provider-signed URL points straight at the stored object and would bypass both.
- **View-only is decided by the server.** `?inline` is a client hint about the disposition; it cannot decide whether the policy applies. A view-only delivery is always served inline, always processed, and its final pass rasterises the pages, so what leaves is a rendition rather than the stored document. Recipients still see the file; they do not receive a working copy of it.
**View-only over a non-PDF.** Stored files are not all PDFs, and the rasterising pass is a PDF operation. For a payload whose type says it is something else (a PNG, a spreadsheet), that pass is skipped and the stored bytes are served: view-only then means only the inline-only disposition, which is what it can mean for a format with no pages to render. Any tool chain the policy configures still runs, and still fails the delivery closed if a step refuses the type. A payload whose type cannot be read at all counts as a PDF, so it is rasterised or refused rather than handed over untouched.
The cached copy is read back through the low-level file store rather than the job-ownership-checked wrapper: the share has already authorised the recipient, and the ownership check would refuse every recipient except the one whose download happened to derive the copy.
### Configuration
Nothing extra. Sharing policies are inert unless the storage sharing feature is on (`storage.sharing.enabled`), and a deployment with no sharing policy behaves exactly as it did before.
## Workflow Share Integration
Signing workflow participants access documents via their own `WorkflowParticipant.shareToken`. No `FileShare` record is created for participants; access control is self-contained in the `WorkflowParticipant` entity.
@@ -336,6 +336,15 @@ public class PolicyController {
* nothing to check.
*/
private void requireAccessibleOutput(Policy policy) {
// An editor policy hands its results back to the workspace the file came from. A stored
// destination would send the run to a folder or bucket instead, leaving the editor's copy
// untouched - and the editor's import would then have nothing to collect.
if (policy.editor().allowed() && !policy.outputIds().isEmpty()) {
throw new ResponseStatusException(
HttpStatus.BAD_REQUEST,
"An editor policy delivers back to the editor and can't also have a"
+ " destination");
}
for (String outputId : policy.outputIds()) {
Source destination =
sourceStore
@@ -393,7 +402,8 @@ public class PolicyController {
policy.steps(),
policy.output(),
policy.outputIds(),
teamId);
teamId,
policy.editor());
}
/** Output secrets never leave the server: reads return the redaction sentinel instead. */
@@ -0,0 +1,34 @@
package stirling.software.proprietary.policy.model;
/**
* How a policy participates in the editor: it fires in the browser as each file passes through,
* rather than being swept from a stored {@code Source} on a trigger.
*
* <p>An object rather than a bare flag so the moment it fires ({@code runOn}) travels with the
* decision, and so later editor-only settings have somewhere to live.
*
* @param allowed whether the editor may run this policy at all
* @param runOn which moment it fires on: {@code "upload"} or {@code "export"}
*/
public record EditorConfig(boolean allowed, String runOn) {
public static final String UPLOAD = "upload";
public static final String EXPORT = "export";
public EditorConfig {
runOn = EXPORT.equals(runOn) ? EXPORT : UPLOAD;
}
/** Not an editor policy: swept server-side, or run only on demand. */
public static EditorConfig disabled() {
return new EditorConfig(false, UPLOAD);
}
public static EditorConfig onUpload() {
return new EditorConfig(true, UPLOAD);
}
public static EditorConfig onExport() {
return new EditorConfig(true, EXPORT);
}
}
@@ -1,6 +1,7 @@
package stirling.software.proprietary.policy.model;
import java.util.List;
import java.util.Optional;
/**
* A stored automation: ordered tool steps, input bindings, and output destinations.
@@ -24,13 +25,29 @@ public record Policy(
List<PipelineStep> steps,
OutputSpec output,
List<String> outputIds,
Long teamId) {
Long teamId,
EditorConfig editor) {
public Policy {
inputs = inputs == null ? List.of() : List.copyOf(inputs);
steps = steps == null ? List.of() : steps;
output = output == null ? OutputSpec.inline() : output;
outputIds = outputIds == null ? List.of() : List.copyOf(outputIds);
editor = editor == null ? EditorConfig.disabled() : editor;
}
/** Without editor participation: a swept or on-demand policy. */
public Policy(
String id,
String name,
String owner,
boolean enabled,
List<PipelineInput> inputs,
List<PipelineStep> steps,
OutputSpec output,
List<String> outputIds,
Long teamId) {
this(id, name, owner, enabled, inputs, steps, output, outputIds, teamId, null);
}
/**
@@ -70,6 +87,14 @@ public record Policy(
return inputs.stream().map(PipelineInput::sourceId).toList();
}
/**
* The moment this policy fires in the editor ("upload" / "export"), or empty when the editor
* does not run it. Legacy blobs are lifted onto {@link EditorConfig} when they are read.
*/
public Optional<String> editorRunOn() {
return editor.allowed() ? Optional.of(editor.runOn()) : Optional.empty();
}
/** The distinct trigger types configured across this policy's inputs (manual inputs aside). */
public List<String> triggerTypes() {
return inputs.stream()
@@ -82,17 +107,20 @@ public record Policy(
/** A copy with the inline output replaced (e.g. resolved for the engine, or migrated). */
public Policy withOutput(OutputSpec resolved) {
return new Policy(id, name, owner, enabled, inputs, steps, resolved, outputIds, teamId);
return new Policy(
id, name, owner, enabled, inputs, steps, resolved, outputIds, teamId, editor);
}
/** A copy under a different owner (e.g. moving a seed off a placeholder name). */
public Policy withOwner(String newOwner) {
return new Policy(id, name, newOwner, enabled, inputs, steps, output, outputIds, teamId);
return new Policy(
id, name, newOwner, enabled, inputs, steps, output, outputIds, teamId, editor);
}
/** A copy referencing the given saved output destinations. */
public Policy withOutputIds(List<String> newOutputIds) {
return new Policy(id, name, owner, enabled, inputs, steps, output, newOutputIds, teamId);
return new Policy(
id, name, owner, enabled, inputs, steps, output, newOutputIds, teamId, editor);
}
/**
@@ -114,10 +114,14 @@ public class PolicyOverviewService {
/**
* Summarise a policy's triggers for the overview row: "manual" when no input is triggered,
* otherwise the distinct trigger types across its inputs (e.g. "folder-watch, schedule").
*
* <p>An editor policy has no wire input to trigger, but it is not manual either - it fires in
* the editor on every upload or export, so it reports that rather than reading as on-demand.
*/
private static String triggerSummary(Policy policy) {
List<String> types = policy.triggerTypes();
return types.isEmpty() ? "manual" : String.join(", ", types);
if (!types.isEmpty()) return String.join(", ", types);
return policy.editorRunOn().map(runOn -> "editor-" + runOn).orElse("manual");
}
private static String outputSummary(OutputSpec output) {
@@ -14,6 +14,7 @@ import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.proprietary.model.TeamCreatedEvent;
import stirling.software.proprietary.policy.model.EditorConfig;
import stirling.software.proprietary.policy.model.OutputSpec;
import stirling.software.proprietary.policy.model.PipelineStep;
import stirling.software.proprietary.policy.model.Policy;
@@ -98,9 +99,8 @@ public class DefaultClassificationPolicySeeder {
static Policy defaultPolicy(Long teamId) {
Map<String, Object> options = new HashMap<>();
options.put("categoryId", CATEGORY);
options.put("runOn", "upload");
options.put("mode", "new_version");
options.put("sources", List.of("editor"));
options.put("sources", List.of());
options.put("scopeTypes", List.of());
options.put("reviewerEmail", "");
return new Policy(
@@ -113,6 +113,9 @@ public class DefaultClassificationPolicySeeder {
List.of(),
List.of(new PipelineStep(CLASSIFY_ENDPOINT, Map.of())),
new OutputSpec("inline", options),
teamId);
List.of(),
teamId,
// Classification runs in the editor on every upload.
EditorConfig.onUpload());
}
}
@@ -107,14 +107,12 @@ public class SourceOverviewService {
}
/**
* Whether a policy runs from the editor. Editor membership is carried in the policy's output
* metadata ({@code output.options.sources}) - a client-side list the editor writes when a
* policy targets it - rather than as a persisted {@code sourceId}, because the editor is
* virtual and has no stored source to reference.
* Whether a policy runs from the editor. Read from the policy's first-class {@link
* stirling.software.proprietary.policy.model.EditorConfig}, never inferred from a sources list
* (the editor is not a real source).
*/
private static boolean runsFromEditor(Policy policy) {
Object sources = policy.output().options().get("sources");
return sources instanceof List<?> list && list.contains(EditorSource.ID);
return policy.editor().allowed();
}
/**
@@ -38,7 +38,8 @@ public class InProcessPolicyStore implements PolicyStore {
policy.steps(),
policy.output(),
policy.outputIds(),
policy.teamId());
policy.teamId(),
policy.editor());
policies.put(id, stored);
// Existing policy keeps its position; a new one appends to the end of its team's queue.
sortOrders.computeIfAbsent(id, key -> nextSortOrder(stored.teamId()));
@@ -3,6 +3,7 @@ package stirling.software.proprietary.policy.store;
import java.util.List;
import java.util.Objects;
import java.util.Optional;
import java.util.Set;
import java.util.UUID;
import org.springframework.stereotype.Service;
@@ -11,8 +12,10 @@ import org.springframework.transaction.annotation.Transactional;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.proprietary.policy.model.EditorConfig;
import stirling.software.proprietary.policy.model.Policy;
import stirling.software.proprietary.policy.model.PolicyBinding;
import stirling.software.proprietary.policy.source.EditorSource;
import tools.jackson.databind.JsonNode;
import tools.jackson.databind.ObjectMapper;
@@ -48,7 +51,8 @@ public class JpaPolicyStore implements PolicyStore {
policy.steps(),
policy.output(),
policy.outputIds(),
policy.teamId());
policy.teamId(),
policy.editor());
PolicyEntity entity = new PolicyEntity();
entity.setId(id);
@@ -148,7 +152,9 @@ public class JpaPolicyStore implements PolicyStore {
// One unreadable row must never abort a bulk read or crash startup.
private Optional<Policy> toPolicy(PolicyEntity entity) {
try {
JsonNode node = upgradeLegacyShape(objectMapper.readTree(entity.getPolicyJson()));
JsonNode node =
liftEditorConfig(
upgradeLegacyShape(objectMapper.readTree(entity.getPolicyJson())));
return Optional.of(objectMapper.treeToValue(node, Policy.class));
} catch (Exception e) {
log.error(
@@ -191,4 +197,61 @@ public class JpaPolicyStore implements PolicyStore {
obj.remove("sourceIds");
return obj;
}
/** Categories whose editor moment defaulted to export before it was stored (see runOn.ts). */
private static final Set<String> EXPORT_BY_DEFAULT = Set.of("security");
/**
* Derive {@code editor} for a blob written before editor participation had its own field, from
* its {@code output.options}: allowed when {@code sources} lists {@code "editor"}, or - for a
* catalogue policy - when there is no {@code sources} list at all (an unnarrowed catalogue
* policy runs in the editor).
*
* <p>Runs on every read, deliberately outside {@link #upgradeLegacyShape}'s early return: a
* blob written after triggers moved onto {@code inputs} but before this field existed still
* needs lifting, and that early return would skip exactly those rows.
*/
private JsonNode liftEditorConfig(JsonNode root) {
if (!(root instanceof ObjectNode obj) || obj.hasNonNull("editor")) {
return root;
}
JsonNode options = obj.path("output").path("options");
String categoryId = text(options, "categoryId");
JsonNode sources = options.get("sources");
boolean listed = sources != null && sources.isArray() && !sources.isEmpty();
boolean allowed;
if (listed) {
// An explicit scope list decides: only the editor's own id puts it on the editor.
allowed = false;
for (JsonNode source : sources) {
if (source.isValueNode() && EditorSource.ID.equals(source.asString())) {
allowed = true;
break;
}
}
} else {
// No list: a catalogue policy ran in the editor by default, but a builder pipeline
// (no category) could not reach the editor at all, so silence is not consent there.
allowed = !categoryId.isBlank();
}
ObjectNode editor = objectMapper.createObjectNode();
editor.put("allowed", allowed);
editor.put("runOn", legacyRunOn(options, categoryId));
obj.set("editor", editor);
return obj;
}
/** The stored moment, or the category default the client applied when none was stored. */
private static String legacyRunOn(JsonNode options, String categoryId) {
String stored = text(options, "runOn");
if (EditorConfig.EXPORT.equals(stored) || EditorConfig.UPLOAD.equals(stored)) {
return stored;
}
return EXPORT_BY_DEFAULT.contains(categoryId) ? EditorConfig.EXPORT : EditorConfig.UPLOAD;
}
private static String text(JsonNode parent, String field) {
JsonNode node = parent.path(field);
return node.isValueNode() ? node.asString() : "";
}
}
@@ -35,6 +35,9 @@ import lombok.extern.slf4j.Slf4j;
import stirling.software.proprietary.audit.AuditEventType;
import stirling.software.proprietary.security.model.User;
import stirling.software.proprietary.service.AuditService;
import stirling.software.proprietary.storage.egress.ShareEgressDecision;
import stirling.software.proprietary.storage.egress.ShareEgressException;
import stirling.software.proprietary.storage.egress.ShareEgressProcessor;
import stirling.software.proprietary.storage.model.FileShare;
import stirling.software.proprietary.storage.model.StoredFile;
import stirling.software.proprietary.storage.model.api.CreateShareLinkRequest;
@@ -59,6 +62,7 @@ public class FileStorageController {
private final FileStorageService fileStorageService;
private final StorageProvider storageProvider;
private final ShareEgressProcessor shareEgressProcessor;
private final AuditService auditService;
@PostMapping(
@@ -105,6 +109,12 @@ public class FileStorageController {
User user = fileStorageService.requireAuthenticatedUser();
StoredFile file = fileStorageService.getAccessibleFile(user, fileId);
fileStorageService.requireReadAccess(user, file);
// A recipient fetching a file shared with them is egress; the owner fetching their own is
// not, and findUserShare returns empty for them.
Optional<FileShare> share = fileStorageService.findUserShare(user, file);
if (share.isPresent()) {
return deliverUnderPolicy(share.get(), file, user, inline);
}
Optional<ResponseEntity<org.springframework.core.io.Resource>> redirect =
tryRedirectToSignedUrl(file, inline);
return redirect.orElseGet(() -> buildFileResponse(file, inline));
@@ -203,11 +213,11 @@ public class FileStorageController {
throw new ResponseStatusException(status, message);
}
fileStorageService.requireReadAccess(share);
fileStorageService.recordShareAccess(share, authentication, inline);
StoredFile file = share.getFile();
Optional<ResponseEntity<org.springframework.core.io.Resource>> redirect =
tryRedirectToSignedUrl(file, inline);
return redirect.orElseGet(() -> buildFileResponse(file, inline));
User accessor =
authentication != null && authentication.getPrincipal() instanceof User user
? user
: null;
return deliverUnderPolicy(share, share.getFile(), accessor, inline, authentication);
}
@GetMapping("/share-links/{token}/metadata")
@@ -263,9 +273,53 @@ public class FileStorageController {
return fileStorageService.listShareAccessResponses(owner, file, token);
}
/** Serves a shared document under the owner team's Sharing policies. */
private ResponseEntity<org.springframework.core.io.Resource> deliverUnderPolicy(
FileShare share, StoredFile file, User accessor, boolean inline) {
return deliverUnderPolicy(share, file, accessor, inline, null);
}
private ResponseEntity<org.springframework.core.io.Resource> deliverUnderPolicy(
FileShare share,
StoredFile file,
User accessor,
boolean inline,
Authentication shareLinkAuth) {
ShareEgressDecision decision = fileStorageService.decideDelivery(share, accessor);
if (!decision.allowed()) {
throw new ShareEgressException(decision);
}
// `inline` is a client hint. Under a view-only policy the server picks the disposition
// instead, so appending ?inline cannot decide whether the policy applies.
boolean servedInline = inline || decision.viewOnly();
// Record the access only once the policy has let it through, so a refused attempt does not
// appear in the owner's access log as a successful view.
if (shareLinkAuth != null) {
fileStorageService.recordShareAccess(share, shareLinkAuth, servedInline);
}
if (!decision.requiresManagedDelivery()) {
Optional<ResponseEntity<org.springframework.core.io.Resource>> redirect =
tryRedirectToSignedUrl(file, servedInline);
if (redirect.isPresent()) {
return redirect.get();
}
return buildFileResponse(file, servedInline);
}
// A signed URL would point straight at the stored object, bypassing both the processed copy
// and the view-only disposition, so managed deliveries always stream through here.
org.springframework.core.io.Resource stored = fileStorageService.loadFile(file);
org.springframework.core.io.Resource served =
shareEgressProcessor.resolve(share, stored, file.getOriginalFilename(), decision);
return buildFileResponse(file, served, servedInline);
}
private ResponseEntity<org.springframework.core.io.Resource> buildFileResponse(
StoredFile file, boolean inline) {
org.springframework.core.io.Resource resource = fileStorageService.loadFile(file);
return buildFileResponse(file, fileStorageService.loadFile(file), inline);
}
private ResponseEntity<org.springframework.core.io.Resource> buildFileResponse(
StoredFile file, org.springframework.core.io.Resource resource, boolean inline) {
if (file.getEncryptionKeyId() != null) {
// Compliance marker: a plaintext copy of encrypted-at-rest content left the platform
// (inline=true is an in-app view; false is a saved download).
@@ -296,7 +350,15 @@ public class FileStorageController {
} catch (IllegalArgumentException ex) {
headers.setContentType(MediaType.APPLICATION_OCTET_STREAM);
}
headers.setContentLength(file.getSizeBytes());
// A processed copy is a different size from the stored original, so take the length from
// the resource actually being served and fall back to the record only if it can't say.
long length = file.getSizeBytes();
try {
length = resource.contentLength();
} catch (IOException e) {
log.debug("Could not size the served resource; using the stored size", e);
}
headers.setContentLength(length);
return ResponseEntity.ok().headers(headers).body(resource);
}
@@ -0,0 +1,174 @@
package stirling.software.proprietary.storage.egress;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.Set;
import stirling.software.proprietary.policy.model.PipelineStep;
import stirling.software.proprietary.policy.model.Policy;
import stirling.software.proprietary.storage.model.ShareAccessRole;
/** A stored sharing policy read as a typed rule; the only place {@code output.options} is read. */
public record EgressRule(
String policyId,
String policyName,
/** Channels this rule governs; empty governs all of them. */
Set<ShareChannel> channels,
/** Ceiling on the access role a share may grant; null leaves the requested role alone. */
ShareAccessRole maxRole,
ExternalRecipients externalRecipients,
/** Email domains counted as internal; empty falls back to the file owner's own domain. */
Set<String> internalDomains,
/** Ceiling on share-link lifetime in days; null leaves the configured expiry alone. */
Integer maxLinkDays,
/** Deny attachment downloads, allowing in-browser viewing only. */
boolean viewOnly,
/** The tool chain applied to the copy that leaves; empty serves the stored bytes as-is. */
List<PipelineStep> steps) {
/** What to do with a recipient outside the organisation. */
public enum ExternalRecipients {
/** No extra treatment — external recipients are handled like internal ones. */
ALLOW,
/** The tightest terms the rule can express: read-only, view-only, shortest link life. */
RESTRICT,
/** Refused outright with a 403 naming the policy. */
BLOCK;
static ExternalRecipients parse(Object raw) {
if (raw == null) {
return RESTRICT;
}
return switch (raw.toString().trim().toLowerCase(Locale.ROOT)) {
case "allow" -> ALLOW;
case "block" -> BLOCK;
default -> RESTRICT;
};
}
}
/** The portal category id a Sharing policy is stored under. */
static final String CATEGORY_SHARING = "sharing";
/** Field keys as persisted by the portal's Sharing policy settings. */
static final String FIELD_DEFAULT_ACCESS = "defaultAccess";
static final String FIELD_EXTERNAL_RECIPIENTS = "externalRecipients";
static final String FIELD_INTERNAL_DOMAINS = "internalDomains";
static final String FIELD_LINK_EXPIRY = "linkExpiry";
static final String FIELD_DOWNLOADS = "downloads";
/**
* Whether this stored policy is a Sharing policy. Egress has no source to hang a trigger on, so
* the portal category the policy was authored under is what marks it, as for classification.
*/
public static boolean governsSharing(Policy policy) {
return CATEGORY_SHARING.equals(optionsOf(policy).get("categoryId"));
}
/** Read a stored policy as an egress rule. */
public static EgressRule from(Policy policy) {
Map<String, Object> options = optionsOf(policy);
Map<String, Object> fields = asMap(options.get("fieldValues"));
return new EgressRule(
policy.id(),
policy.name(),
parseChannels(options.get("sources")),
parseMaxRole(fields.get(FIELD_DEFAULT_ACCESS)),
ExternalRecipients.parse(fields.get(FIELD_EXTERNAL_RECIPIENTS)),
parseDomains(fields.get(FIELD_INTERNAL_DOMAINS)),
parseLinkDays(fields.get(FIELD_LINK_EXPIRY)),
"viewOnly".equalsIgnoreCase(string(fields.get(FIELD_DOWNLOADS))),
policy.steps() == null ? List.of() : List.copyOf(policy.steps()));
}
/** Whether this rule governs the given channel. */
public boolean covers(ShareChannel channel) {
return channels.isEmpty() || channels.contains(channel);
}
private static Map<String, Object> optionsOf(Policy policy) {
return policy.output() == null ? Map.of() : policy.output().options();
}
private static Set<ShareChannel> parseChannels(Object raw) {
Set<ShareChannel> parsed = new LinkedHashSet<>();
for (String id : asStrings(raw)) {
ShareChannel channel = ShareChannel.fromId(id);
if (channel != null) {
parsed.add(channel);
}
}
return Set.copyOf(parsed);
}
/** "restricted" is the portal's tightest role; "inherit"/unknown means no ceiling. */
private static ShareAccessRole parseMaxRole(Object raw) {
String value = string(raw);
if (value == null) {
return null;
}
return switch (value.trim().toLowerCase(Locale.ROOT)) {
case "restricted", "viewer" -> ShareAccessRole.VIEWER;
case "commenter" -> ShareAccessRole.COMMENTER;
case "editor" -> ShareAccessRole.EDITOR;
default -> null;
};
}
private static Integer parseLinkDays(Object raw) {
String value = string(raw);
if (value == null) {
return null;
}
return switch (value.trim().toLowerCase(Locale.ROOT)) {
case "oneday" -> 1;
case "threedays" -> 3;
case "sevendays" -> 7;
case "thirtydays" -> 30;
default -> null;
};
}
/** Domains are compared case-insensitively and tolerate a leading "@" or "*." in the input. */
private static Set<String> parseDomains(Object raw) {
Set<String> domains = new LinkedHashSet<>();
for (String entry : asStrings(raw)) {
String domain = entry.trim().toLowerCase(Locale.ROOT);
if (domain.startsWith("@")) {
domain = domain.substring(1);
}
if (domain.startsWith("*.")) {
domain = domain.substring(2);
}
if (!domain.isEmpty()) {
domains.add(domain);
}
}
return Set.copyOf(domains);
}
private static List<String> asStrings(Object raw) {
if (!(raw instanceof Iterable<?> items)) {
return List.of();
}
List<String> values = new java.util.ArrayList<>();
for (Object item : items) {
if (item != null && !item.toString().isBlank()) {
values.add(item.toString());
}
}
return List.copyOf(values);
}
@SuppressWarnings("unchecked")
private static Map<String, Object> asMap(Object raw) {
return raw instanceof Map<?, ?> map ? (Map<String, Object>) map : Map.of();
}
private static String string(Object raw) {
return raw == null || raw.toString().isBlank() ? null : raw.toString();
}
}
@@ -0,0 +1,40 @@
package stirling.software.proprietary.storage.egress;
import java.util.Locale;
/** How a document is leaving. Ids are persisted in a policy's {@code sources}. */
public enum ShareChannel {
/** Shared directly with another registered user of this deployment. */
USER_SHARE("userShare"),
/** A token share link was minted; anyone holding it can open the document. */
SHARE_LINK("shareLink"),
/** Shared to an email address, which mails a link to a recipient who may have no account. */
EMAIL_SHARE("emailShare");
private final String id;
ShareChannel(String id) {
this.id = id;
}
public String id() {
return id;
}
/** The channel for a stored id, or null when the id names something else (e.g. "editor"). */
public static ShareChannel fromId(String id) {
if (id == null) {
return null;
}
String normalized = id.trim().toLowerCase(Locale.ROOT);
for (ShareChannel channel : values()) {
if (channel.id.toLowerCase(Locale.ROOT).equals(normalized)) {
return channel;
}
}
return null;
}
}
@@ -0,0 +1,66 @@
package stirling.software.proprietary.storage.egress;
import java.util.List;
import stirling.software.proprietary.storage.model.ShareAccessRole;
/** What the sharing policies decided about one egress attempt; never null. */
public record ShareEgressDecision(
boolean allowed,
/** Human-readable reason a blocked share was refused; null when allowed. */
String reason,
/** The role the share may actually grant, after any policy ceiling. */
ShareAccessRole role,
/** Ceiling on link lifetime in days, or null to keep the configured expiry. */
Integer maxLinkDays,
/** Deny attachment downloads; the document may still be viewed in the browser. */
boolean viewOnly,
/** Whether the recipient was classified as outside the organisation. */
boolean external,
/** The policy that decided, for the audit trail and the refusal message; null if none. */
String policyId,
String policyName,
/** Policies to run the copy through, in order. Ids, so each is its own billed run. */
List<String> transformPolicyIds,
/** Identity of the copy this would produce; a differently stamped cache is stale. */
String transformFingerprint) {
public ShareEgressDecision {
transformPolicyIds =
transformPolicyIds == null ? List.of() : List.copyOf(transformPolicyIds);
}
/** No sharing policy applies: the share proceeds exactly as the caller asked. */
public static ShareEgressDecision unrestricted(ShareAccessRole requestedRole) {
return new ShareEgressDecision(
true, null, requestedRole, null, false, false, null, null, List.of(), null);
}
/** Refused, naming the policy so the caller can see which rule stopped them. */
public static ShareEgressDecision blocked(EgressRule rule, String reason) {
return new ShareEgressDecision(
false,
reason,
null,
null,
false,
true,
rule.policyId(),
rule.policyName(),
List.of(),
null);
}
/** Whether a policy tool chain runs over the copy that leaves. */
public boolean transforms() {
return !transformPolicyIds.isEmpty();
}
/**
* Must stream through the egress processor: a signed URL would hand over the stored original,
* and a view-only copy is rasterised rather than served as-is.
*/
public boolean requiresManagedDelivery() {
return viewOnly || transforms();
}
}
@@ -0,0 +1,26 @@
package stirling.software.proprietary.storage.egress;
import org.springframework.http.HttpStatus;
import org.springframework.web.server.ResponseStatusException;
/** A sharing policy refused this egress; names the rule that stopped them. */
public class ShareEgressException extends ResponseStatusException {
private final transient String policyId;
public ShareEgressException(ShareEgressDecision decision) {
super(HttpStatus.FORBIDDEN, message(decision));
this.policyId = decision.policyId();
}
public String getPolicyId() {
return policyId;
}
private static String message(ShareEgressDecision decision) {
String reason = decision.reason() == null ? "Sharing is not permitted" : decision.reason();
return decision.policyName() == null
? reason
: reason + " (blocked by the \"" + decision.policyName() + "\" policy)";
}
}
@@ -0,0 +1,233 @@
package stirling.software.proprietary.storage.egress;
import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.ArrayList;
import java.util.HexFormat;
import java.util.List;
import java.util.Locale;
import java.util.Set;
import org.springframework.stereotype.Service;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.proprietary.policy.model.Policy;
import stirling.software.proprietary.policy.store.PolicyStore;
import stirling.software.proprietary.security.model.User;
import stirling.software.proprietary.storage.model.FileShare;
import stirling.software.proprietary.storage.model.ShareAccessRole;
import stirling.software.proprietary.storage.model.StoredFile;
import tools.jackson.databind.ObjectMapper;
/** Evaluates the file owner's team's Sharing policies; several compose to the strictest outcome. */
@Slf4j
@Service
@RequiredArgsConstructor
public class ShareEgressPolicyService {
private final PolicyStore policyStore;
private final ObjectMapper objectMapper;
/** Decide a share that is about to be granted. */
public ShareEgressDecision evaluateGrant(
ShareChannel channel,
StoredFile file,
User owner,
String recipient,
ShareAccessRole requestedRole) {
return evaluate(channel, file, owner, recipient, requestedRole);
}
/** Decide a delivery: channel inferred from the share row, recipient is whoever is fetching. */
public ShareEgressDecision evaluateDelivery(FileShare share, User accessor) {
StoredFile file = share.getFile();
User owner = file == null ? null : file.getOwner();
return evaluate(
channelOf(share),
file,
owner,
accessor == null ? null : accessor.getUsername(),
share.getAccessRole());
}
/** The channel stamped when the share was granted; older rows fall back to their shape. */
private static ShareChannel channelOf(FileShare share) {
if (share.getSharedWithUser() != null) {
// A row naming a user is a user share whatever it was stamped, so typing that user's
// email address cannot turn their share into an email share at delivery.
return ShareChannel.USER_SHARE;
}
return share.getEgressChannel() != null
? share.getEgressChannel()
: ShareChannel.SHARE_LINK;
}
private ShareEgressDecision evaluate(
ShareChannel channel,
StoredFile file,
User owner,
String recipient,
ShareAccessRole requestedRole) {
List<EgressRule> rules = rulesFor(owner, channel);
if (rules.isEmpty()) {
return ShareEgressDecision.unrestricted(requestedRole);
}
ShareAccessRole role = requestedRole;
Integer maxLinkDays = null;
boolean viewOnly = false;
boolean external = false;
String decidingPolicyId = null;
String decidingPolicyName = null;
List<String> transformPolicyIds = new ArrayList<>();
StringBuilder fingerprintSource = new StringBuilder();
for (EgressRule rule : rules) {
boolean ruleSaysExternal = isExternal(recipient, owner, rule);
external |= ruleSaysExternal;
if (ruleSaysExternal
&& rule.externalRecipients() == EgressRule.ExternalRecipients.BLOCK) {
log.info(
"Sharing policy {} blocked egress of file {} to an external recipient",
rule.policyId(),
file == null ? null : file.getId());
return ShareEgressDecision.blocked(
rule, "This document may not be shared outside your organisation");
}
// "Restrict" is the tightest terms the rule can express: read-only, no download, and
// the shortest link life, so an external party gets a look rather than a working copy.
boolean restrictExternal =
ruleSaysExternal
&& rule.externalRecipients() == EgressRule.ExternalRecipients.RESTRICT;
role = tightest(role, restrictExternal ? ShareAccessRole.VIEWER : rule.maxRole());
// Integer.valueOf, not a bare 1: a mixed int/Integer ternary unboxes, so a rule with no
// configured expiry would NPE here.
maxLinkDays =
shortest(
maxLinkDays,
restrictExternal ? Integer.valueOf(1) : rule.maxLinkDays());
viewOnly |= rule.viewOnly() || restrictExternal;
if (!rule.steps().isEmpty()) {
transformPolicyIds.add(rule.policyId());
fingerprintSource
.append(rule.policyId())
.append('|')
.append(objectMapper.writeValueAsString(rule.steps()))
.append('\n');
}
decidingPolicyId = rule.policyId();
decidingPolicyName = rule.policyName();
}
if (decidingPolicyId == null) {
return ShareEgressDecision.unrestricted(requestedRole);
}
// View-only rasterises the copy even with no tool chain, so it is processed too and needs a
// fingerprint of its own to cache under.
if (viewOnly) {
fingerprintSource.append("viewOnly\n");
}
String fingerprint =
transformPolicyIds.isEmpty() && !viewOnly
? null
: fingerprint(fingerprintSource.toString(), sourceVersion(file));
return new ShareEgressDecision(
true,
null,
role,
maxLinkDays,
viewOnly,
external,
decidingPolicyId,
decidingPolicyName,
List.copyOf(transformPolicyIds),
fingerprint);
}
/** The owner's team's enabled sharing policies that govern this channel, in run order. */
private List<EgressRule> rulesFor(User owner, ShareChannel channel) {
Long ownerTeamId =
owner != null && owner.getTeam() != null ? owner.getTeam().getId() : null;
List<EgressRule> rules = new ArrayList<>();
for (Policy policy : policyStore.findByTeam(ownerTeamId)) {
if (!policy.enabled() || !EgressRule.governsSharing(policy)) {
continue;
}
EgressRule rule = EgressRule.from(policy);
if (rule.covers(channel)) {
rules.add(rule);
}
}
return rules;
}
/** Inside = the rule's domains, else the owner's. No domain reads as internal. */
private static boolean isExternal(String recipient, User owner, EgressRule rule) {
String recipientDomain = domainOf(recipient);
if (recipientDomain == null) {
return false;
}
Set<String> internal = rule.internalDomains();
if (internal.isEmpty()) {
String ownerDomain = owner == null ? null : domainOf(owner.getUsername());
return ownerDomain != null && !ownerDomain.equals(recipientDomain);
}
return !internal.contains(recipientDomain);
}
private static String domainOf(String address) {
if (address == null) {
return null;
}
int at = address.lastIndexOf('@');
if (at < 0 || at == address.length() - 1) {
return null;
}
return address.substring(at + 1).trim().toLowerCase(Locale.ROOT);
}
/** The more restrictive of two roles; a null ceiling leaves the current role alone. */
private static ShareAccessRole tightest(ShareAccessRole current, ShareAccessRole ceiling) {
if (ceiling == null) {
return current;
}
if (current == null) {
return ceiling;
}
// Declaration order is EDITOR, COMMENTER, VIEWER — least to most restrictive.
return current.ordinal() >= ceiling.ordinal() ? current : ceiling;
}
private static Integer shortest(Integer current, Integer candidate) {
if (candidate == null) {
return current;
}
return current == null ? candidate : Math.min(current, candidate);
}
/** Changes whenever the document is replaced, so a cached processed copy goes stale with it. */
private static String sourceVersion(StoredFile file) {
if (file == null) {
return "none";
}
return file.getId() + ":" + file.getUpdatedAt() + ":" + file.getSizeBytes();
}
private static String fingerprint(String chain, String sourceVersion) {
try {
byte[] digest =
MessageDigest.getInstance("SHA-256")
.digest((chain + sourceVersion).getBytes(StandardCharsets.UTF_8));
return HexFormat.of().formatHex(digest);
} catch (NoSuchAlgorithmException e) {
throw new IllegalStateException("SHA-256 is required by the Java platform", e);
}
}
}
@@ -0,0 +1,302 @@
package stirling.software.proprietary.storage.egress;
import java.io.IOException;
import java.io.InputStream;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.Objects;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
import org.springframework.core.io.AbstractResource;
import org.springframework.core.io.ByteArrayResource;
import org.springframework.core.io.Resource;
import org.springframework.stereotype.Service;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.cluster.FileStore;
import stirling.software.common.model.job.ResultFile;
import stirling.software.proprietary.policy.engine.PolicyEngine;
import stirling.software.proprietary.policy.engine.PolicyRunHandle;
import stirling.software.proprietary.policy.model.OutputSpec;
import stirling.software.proprietary.policy.model.PipelineStep;
import stirling.software.proprietary.policy.model.Policy;
import stirling.software.proprietary.policy.model.PolicyInputs;
import stirling.software.proprietary.policy.model.PolicyRun;
import stirling.software.proprietary.policy.model.PolicyRunStatus;
import stirling.software.proprietary.policy.progress.PolicyProgressListener;
import stirling.software.proprietary.policy.store.PolicyStore;
import stirling.software.proprietary.storage.model.FileShare;
import stirling.software.proprietary.storage.repository.FileShareRepository;
/** Produces the processed copy a recipient receives; cached per share, and fails closed. */
@Slf4j
@Service
@RequiredArgsConstructor
public class ShareEgressProcessor {
/** Long enough for a rasterising watermark, short enough to surface a wedged run. */
private static final long TRANSFORM_TIMEOUT_SECONDS = 120;
/** Renders every page to an image, so a view-only copy carries no reusable text or objects. */
private static final PipelineStep RASTERISE =
new PipelineStep("/api/v1/misc/flatten", Map.of("flattenOnlyForms", false));
private final PolicyStore policyStore;
private final PolicyEngine policyEngine;
/**
* The low-level store, not {@code FileStorage}: the share has already authorised this
* recipient, and the job-ownership gate would refuse every recipient but the one whose download
* derived the cached copy.
*/
private final FileStore fileStore;
private final FileShareRepository fileShareRepository;
/**
* The bytes to serve: cached copy, a fresh one, or the original when nothing processes it.
*
* @param filename what the document is called; storage providers hand back unnamed resources
* and the tool chain decides what it accepts from the extension.
*/
public Resource resolve(
FileShare share, Resource original, String filename, ShareEgressDecision decision) {
if (!decision.requiresManagedDelivery()) {
return original;
}
String cached = cachedFileId(share, decision);
if (cached != null) {
try {
return toResource(cached, filename);
} catch (IOException | RuntimeException e) {
log.warn(
"Cached processed copy {} could not be read; re-deriving: {}",
cached,
e.getMessage());
}
}
return derive(share, original, filename, decision);
}
/** The still-valid cached copy for this decision, or null if absent, stale, or swept away. */
private String cachedFileId(FileShare share, ShareEgressDecision decision) {
String fileId = share.getEgressFileId();
if (fileId == null
|| !Objects.equals(decision.transformFingerprint(), share.getEgressFingerprint())) {
return null;
}
return fileStore.exists(fileId) ? fileId : null;
}
/** Runs each policy's chain on the previous one's output; records the result on the share. */
private Resource derive(
FileShare share, Resource original, String filename, ShareEgressDecision decision) {
Resource current = named(original, filename);
String fileId = null;
for (String policyId : decision.transformPolicyIds()) {
Policy policy = policyStore.get(policyId).orElse(null);
if (policy == null || policy.steps().isEmpty()) {
// Deleted between the decision and here; nothing to apply for this link in the
// chain, so carry the current bytes forward.
continue;
}
fileId = runChain(policy, current, decision);
current = read(fileId, filename, decision);
}
if (decision.viewOnly() && rasterisable(filename)) {
// View-only means no working copy leaves, so the bytes are rasterised whatever
// disposition the client asked for.
fileId = runChain(rasterisingPolicy(decision), current, decision);
current = read(fileId, filename, decision);
}
if (fileId == null) {
if (decision.viewOnly()) {
// Nothing here to rasterise, so the stored bytes are what the recipient sees;
// view-only still binds as the inline-only disposition the caller enforces.
log.debug(
"View-only delivery of a non-PDF payload {} serves it as stored", filename);
return current;
}
throw new ShareEgressException(
new ShareEgressDecision(
false,
"The sharing policy that governs this document is no longer available",
null,
null,
false,
decision.external(),
decision.policyId(),
decision.policyName(),
List.of(),
null));
}
stampCache(share, decision, fileId);
return current;
}
/**
* Whether the rasterising pass can apply at all: only a PDF has pages to render. An unknown
* type counts as one, so a view-only delivery fails closed rather than handing over the stored
* document.
*/
private static boolean rasterisable(String filename) {
String extension = extensionOf(filename);
return extension == null || "pdf".equals(extension);
}
private static String extensionOf(String filename) {
if (filename == null) {
return null;
}
int dot = filename.lastIndexOf('.');
if (dot < 0 || dot == filename.length() - 1) {
return null;
}
return filename.substring(dot + 1).toLowerCase(Locale.ROOT);
}
/** The rasterising pass, attributed and billed to the policy that asked for view-only. */
private Policy rasterisingPolicy(ShareEgressDecision decision) {
Policy deciding =
decision.policyId() == null
? null
: policyStore.get(decision.policyId()).orElse(null);
return new Policy(
decision.policyId(),
deciding != null ? deciding.name() : decision.policyName(),
deciding != null ? deciding.owner() : null,
true,
List.of(),
List.of(RASTERISE),
OutputSpec.inline(),
deciding != null ? deciding.teamId() : null);
}
/** Attributed to the policy and billed to its owner, not to whoever downloads. */
private String runChain(Policy policy, Resource input, ShareEgressDecision decision) {
PolicyRunHandle handle =
policyEngine.runPolicy(
policy, PolicyInputs.of(List.of(input)), PolicyProgressListener.NOOP);
PolicyRun run;
try {
run = handle.completion().get(TRANSFORM_TIMEOUT_SECONDS, TimeUnit.SECONDS);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
throw refuse(decision, e);
} catch (ExecutionException | TimeoutException e) {
policyEngine.cancel(handle.runId());
throw refuse(decision, e);
}
if (run.getStatus() != PolicyRunStatus.COMPLETED || run.getOutputs().isEmpty()) {
log.warn(
"Sharing policy {} could not process the outgoing copy (run {} ended {}): {}",
policy.id(),
handle.runId(),
run.getStatus(),
run.getError());
throw refuse(decision, null);
}
ResultFile output = run.getOutputs().get(0);
return output.getFileId();
}
/** Best-effort: a failed stamp costs a re-derive, so it must not fail the download. */
private void stampCache(FileShare share, ShareEgressDecision decision, String fileId) {
try {
share.setEgressFileId(fileId);
share.setEgressFingerprint(decision.transformFingerprint());
fileShareRepository.save(share);
} catch (RuntimeException e) {
log.warn(
"Could not cache the processed copy for share {}: {}",
share.getId(),
e.getMessage());
}
}
private Resource read(String fileId, String filename, ShareEgressDecision decision) {
try {
return toResource(fileId, filename);
} catch (IOException e) {
throw refuse(decision, e);
}
}
private Resource toResource(String fileId, String filename) throws IOException {
byte[] bytes = fileStore.retrieveBytes(fileId);
return new ByteArrayResource(bytes) {
@Override
public String getFilename() {
return filename;
}
};
}
/** The stored bytes under the document's own name; providers hand back unnamed resources. */
private static Resource named(Resource resource, String filename) {
if (filename == null || filename.equals(resource.getFilename())) {
return resource;
}
return new NamedResource(resource, filename);
}
/** Renames without reading: the stored resource may decrypt on open, or be single-use. */
private static final class NamedResource extends AbstractResource {
private final Resource delegate;
private final String filename;
private NamedResource(Resource delegate, String filename) {
this.delegate = delegate;
this.filename = filename;
}
@Override
public InputStream getInputStream() throws IOException {
return delegate.getInputStream();
}
@Override
public long contentLength() throws IOException {
return delegate.contentLength();
}
@Override
public boolean exists() {
return delegate.exists();
}
@Override
public String getFilename() {
return filename;
}
@Override
public String getDescription() {
return delegate.getDescription();
}
}
private static ShareEgressException refuse(ShareEgressDecision decision, Throwable cause) {
if (cause != null) {
log.warn("Egress processing failed for policy {}", decision.policyId(), cause);
}
return new ShareEgressException(
new ShareEgressDecision(
false,
"This document could not be prepared for sharing, so it was not released",
null,
null,
false,
decision.external(),
decision.policyId(),
decision.policyName(),
List.of(),
null));
}
}
@@ -24,6 +24,7 @@ import lombok.NoArgsConstructor;
import lombok.Setter;
import stirling.software.proprietary.security.model.User;
import stirling.software.proprietary.storage.egress.ShareChannel;
/** Represents a file sharing relationship between a file and a user or token. */
@Entity
@@ -71,6 +72,19 @@ public class FileShare implements Serializable {
@Column(name = "expires_at")
private LocalDateTime expiresAt;
/** The channel this share was granted through, so delivery is judged as the grant was. */
@Enumerated(EnumType.STRING)
@Column(name = "egress_channel", length = 32)
private ShareChannel egressChannel;
/** Job-storage id of the processed copy; cache only, re-derived if swept. */
@Column(name = "egress_file_id")
private String egressFileId;
/** Identity of that copy (policies + steps + source version); a mismatch means it is stale. */
@Column(name = "egress_fingerprint", length = 64)
private String egressFingerprint;
@CreationTimestamp
@Column(name = "created_at", updatable = false)
private LocalDateTime createdAt;
@@ -23,6 +23,18 @@ public interface FileShareRepository extends JpaRepository<FileShare, Long> {
+ "WHERE s.shareToken = :shareToken")
Optional<FileShare> findByShareTokenWithFile(@Param("shareToken") String shareToken);
/**
* With everything an egress decision reads. open-in-view is off, so a share handed in from an
* earlier transaction is detached and would throw on file/owner/team.
*/
@Query(
"SELECT s FROM FileShare s "
+ "JOIN FETCH s.file f "
+ "LEFT JOIN FETCH f.owner o "
+ "LEFT JOIN FETCH o.team "
+ "WHERE s.id = :id")
Optional<FileShare> findByIdForEgress(@Param("id") Long id);
@Query("SELECT s FROM FileShare s WHERE s.file = :file AND s.shareToken IS NOT NULL")
List<FileShare> findShareLinks(@Param("file") StoredFile file);
@@ -34,6 +34,10 @@ import stirling.software.proprietary.security.model.User;
import stirling.software.proprietary.security.service.EmailService;
import stirling.software.proprietary.storage.crypto.StorageEncryptionErrors;
import stirling.software.proprietary.storage.crypto.StorageKeyRevokedException;
import stirling.software.proprietary.storage.egress.ShareChannel;
import stirling.software.proprietary.storage.egress.ShareEgressDecision;
import stirling.software.proprietary.storage.egress.ShareEgressException;
import stirling.software.proprietary.storage.egress.ShareEgressPolicyService;
import stirling.software.proprietary.storage.model.FileShare;
import stirling.software.proprietary.storage.model.FileShareAccess;
import stirling.software.proprietary.storage.model.FileShareAccessType;
@@ -70,6 +74,7 @@ public class FileStorageService {
private final StorageProvider storageProvider;
private final Optional<EmailService> emailService;
private final StorageCleanupEntryRepository storageCleanupEntryRepository;
private final ShareEgressPolicyService shareEgressPolicyService;
public void ensureStorageEnabled() {
if (!applicationProperties.getSecurity().isEnableLogin()) {
@@ -542,6 +547,26 @@ public class FileStorageService {
boolean isEmail = isEmailAddress(normalizedUsername);
Optional<User> targetUserOpt = userRepository.findByUsernameIgnoreCase(normalizedUsername);
// Egress gate, before anything is written: a Sharing policy may refuse this recipient
// outright, or cap the access they are granted.
// Usernames are commonly email addresses, so a registered recipient is a user share however
// it was typed; only a link mailed to someone with no account is an email share.
ShareChannel channel =
isEmail && targetUserOpt.isEmpty()
? ShareChannel.EMAIL_SHARE
: ShareChannel.USER_SHARE;
ShareEgressDecision decision =
requireEgressAllowed(channel, file, owner, normalizedUsername, role);
ShareAccessRole effectiveRole = decision.role();
// Sharing with a registered user by email also mails a link, and that link is an email
// share in its own right, so it is judged as one before anything is written.
ShareEgressDecision linkDecision =
isEmail && channel != ShareChannel.EMAIL_SHARE
? requireEgressAllowed(
ShareChannel.EMAIL_SHARE, file, owner, normalizedUsername, role)
: decision;
if (targetUserOpt.isPresent()) {
User targetUser = targetUserOpt.get();
if (targetUser.getId().equals(owner.getId())) {
@@ -554,7 +579,8 @@ public class FileStorageService {
.findByFileAndSharedWithUser(file, targetUser)
.map(
existingShare -> {
existingShare.setAccessRole(role);
existingShare.setAccessRole(effectiveRole);
existingShare.setEgressChannel(channel);
return fileShareRepository.save(existingShare);
})
.orElseGet(
@@ -562,7 +588,8 @@ public class FileStorageService {
FileShare newShare = new FileShare();
newShare.setFile(file);
newShare.setSharedWithUser(targetUser);
newShare.setAccessRole(role);
newShare.setAccessRole(effectiveRole);
newShare.setEgressChannel(channel);
return fileShareRepository.save(newShare);
});
@@ -576,10 +603,16 @@ public class FileStorageService {
HttpStatus.BAD_REQUEST,
"Share links must be enabled for email sharing");
}
String shareLinkUrl = null;
FileShare linkShare = createShareLink(owner, file, role);
shareLinkUrl = buildShareLinkUrl(linkShare);
sendShareNotification(owner, file, normalizedUsername, role, shareLinkUrl);
// Already judged as an email share above; don't re-evaluate it as a link share.
FileShare linkShare =
mintShareLink(
file, linkDecision.role(), linkDecision, ShareChannel.EMAIL_SHARE);
sendShareNotification(
owner,
file,
normalizedUsername,
linkDecision.role(),
buildShareLinkUrl(linkShare));
}
return share;
@@ -596,8 +629,10 @@ public class FileStorageService {
HttpStatus.BAD_REQUEST, "Share links must be enabled for email sharing");
}
FileShare linkShare = createShareLink(owner, file, role);
sendShareNotification(owner, file, normalizedUsername, role, buildShareLinkUrl(linkShare));
FileShare linkShare =
mintShareLink(file, linkDecision.role(), linkDecision, ShareChannel.EMAIL_SHARE);
sendShareNotification(
owner, file, normalizedUsername, linkDecision.role(), buildShareLinkUrl(linkShare));
return linkShare;
}
@@ -640,15 +675,66 @@ public class FileStorageService {
if (!isOwner(file, owner)) {
throw new ResponseStatusException(HttpStatus.FORBIDDEN, "Only the owner can share");
}
// A bare link has no named recipient, so a policy can only judge it as a channel — the
// per-recipient rules bite when someone opens it (see decideDelivery).
ShareEgressDecision decision =
requireEgressAllowed(ShareChannel.SHARE_LINK, file, owner, null, role);
return mintShareLink(file, decision.role(), decision, ShareChannel.SHARE_LINK);
}
/** Writes the link row for an egress decision that has already been made. */
private FileShare mintShareLink(
StoredFile file,
ShareAccessRole role,
ShareEgressDecision decision,
ShareChannel channel) {
FileShare share = new FileShare();
share.setFile(file);
share.setShareToken(UUID.randomUUID().toString());
share.setAccessRole(role);
share.setExpiresAt(resolveShareLinkExpiration());
// Stamped so the link an email share mints is still judged as an email share when it is
// opened; otherwise a policy narrowed to that channel would never bite at delivery.
share.setEgressChannel(channel);
share.setExpiresAt(resolveShareLinkExpiration(decision));
return fileShareRepository.save(share);
}
/** Evaluates the owner team's Sharing policies for a grant; throws if any policy blocks. */
private ShareEgressDecision requireEgressAllowed(
ShareChannel channel,
StoredFile file,
User owner,
String recipient,
ShareAccessRole role) {
ShareEgressDecision decision =
shareEgressPolicyService.evaluateGrant(channel, file, owner, recipient, role);
if (!decision.allowed()) {
throw new ShareEgressException(decision);
}
return decision;
}
/** The egress decision for a delivery; an owner fetching their own file never reaches here. */
public ShareEgressDecision decideDelivery(FileShare share, User accessor) {
return shareEgressPolicyService.evaluateDelivery(attachedForEgress(share), accessor);
}
/** The same row re-read inside this transaction with file, owner and team already fetched. */
private FileShare attachedForEgress(FileShare share) {
if (share == null || share.getId() == null) {
return share;
}
return fileShareRepository.findByIdForEgress(share.getId()).orElse(share);
}
/** The share row backing a non-owner's access to a file, if there is one. */
public Optional<FileShare> findUserShare(User user, StoredFile file) {
if (user == null || file == null || isOwner(file, user)) {
return Optional.empty();
}
return fileShareRepository.findByFileAndSharedWithUser(file, user);
}
public void revokeShareLink(User owner, StoredFile file, String token) {
ensureStorageEnabled();
if (!isOwner(file, owner)) {
@@ -1098,6 +1184,17 @@ public class FileStorageService {
return LocalDateTime.now().plus(days, ChronoUnit.DAYS);
}
/** The configured expiry, tightened to the policy ceiling; a policy can only shorten it. */
private LocalDateTime resolveShareLinkExpiration(ShareEgressDecision decision) {
LocalDateTime configured = resolveShareLinkExpiration();
Integer maxDays = decision.maxLinkDays();
if (maxDays == null) {
return configured;
}
LocalDateTime capped = LocalDateTime.now().plus(maxDays, ChronoUnit.DAYS);
return configured == null || configured.isAfter(capped) ? capped : configured;
}
private boolean isShareLinkExpired(FileShare share) {
if (share == null || share.getExpiresAt() == null) {
return false;
@@ -15,6 +15,7 @@ import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.service.UserServiceInterface;
import stirling.software.proprietary.policy.config.PolicyAccessGuard;
import stirling.software.proprietary.policy.config.PolicyManagementAuthority;
import stirling.software.proprietary.policy.model.EditorConfig;
import stirling.software.proprietary.policy.model.OutputSpec;
import stirling.software.proprietary.policy.model.PipelineInput;
import stirling.software.proprietary.policy.model.PipelineStep;
@@ -223,6 +224,44 @@ class PolicyOverviewServiceTest {
teamId));
}
@Test
void editorPolicyReportsItsRunMomentRatherThanReadingAsManual() {
policyStore.save(
new Policy(
null,
"Editor flatten",
"owner",
true,
List.of(),
List.of(new PipelineStep("/api/v1/misc/flatten", Map.of())),
OutputSpec.inline(),
List.of(),
1L,
EditorConfig.onUpload()));
PolicyView view = find(service.overview(), "Editor flatten");
assertEquals("editor-upload", view.trigger());
}
@Test
void sweptPolicyWithNoTriggeredInputIsStillManual() {
policyStore.save(
new Policy(
null,
"Swept compress",
"owner",
true,
List.of(),
List.of(new PipelineStep("/api/v1/misc/compress-pdf", Map.of())),
OutputSpec.inline(),
1L));
PolicyView view = find(service.overview(), "Swept compress");
assertEquals("manual", view.trigger());
}
private static PolicyView find(PoliciesOverviewResponse response, String name) {
return response.pipelines().stream()
.filter(view -> view.name().equals(name))
@@ -64,14 +64,30 @@ class DefaultClassificationPolicySeederTest {
assertThat(policy.teamId()).isEqualTo(7L);
assertThat(policy.output().type()).isEqualTo("inline");
assertThat(policy.output().options().get("categoryId")).isEqualTo("classification");
assertThat(policy.output().options().get("runOn")).isEqualTo("upload");
assertThat(policy.output().options().get("mode")).isEqualTo("new_version");
assertThat(policy.output().options().get("sources")).isEqualTo(List.of("editor"));
// Editor participation is the policy's own flag, not a marker in the output options.
assertThat(policy.editor().allowed()).isTrue();
assertThat(policy.editor().runOn()).isEqualTo("upload");
assertThat(policy.steps()).hasSize(1);
assertThat(policy.steps().get(0).operation())
.isEqualTo("/api/v1/ai/tools/classify-and-label");
}
@Test
void marksEditorParticipationOnEditorConfigAndSeedsNoSources() {
when(policyStore.findByTeam(7L)).thenReturn(List.of());
seeder().onTeamCreated(new TeamCreatedEvent(7L, "Acme"));
ArgumentCaptor<Policy> saved = ArgumentCaptor.forClass(Policy.class);
verify(policyStore).save(saved.capture());
Policy policy = saved.getValue();
// Editor participation is on EditorConfig, not the sources list; the seed carries no
// sources.
assertThat(policy.editor().allowed()).isTrue();
assertThat(policy.output().options().get("sources")).isEqualTo(List.of());
}
@Test
void doesNotSeedWhenAClassificationPolicyAlreadyExists() {
when(policyStore.findByTeam(7L)).thenReturn(List.of(classificationPolicy(7L)));
@@ -15,6 +15,7 @@ import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.service.UserServiceInterface;
import stirling.software.proprietary.policy.config.PolicyAccessGuard;
import stirling.software.proprietary.policy.config.PolicyManagementAuthority;
import stirling.software.proprietary.policy.model.EditorConfig;
import stirling.software.proprietary.policy.model.OutputSpec;
import stirling.software.proprietary.policy.model.PipelineInput;
import stirling.software.proprietary.policy.model.PipelineStep;
@@ -222,9 +223,7 @@ class SourceOverviewServiceTest {
OutputSpec.inline()));
}
/**
* A policy that targets the editor: membership rides in its output metadata, not a sourceId.
*/
/** A policy that targets the editor: membership on its {@link EditorConfig}, not a sourceId. */
private void editorPolicy(String name) {
policyStore.save(
new Policy(
@@ -234,7 +233,10 @@ class SourceOverviewServiceTest {
true,
List.of(),
List.of(new PipelineStep("/api/v1/misc/compress-pdf", Map.of())),
new OutputSpec("inline", Map.of("sources", List.of("editor")))));
OutputSpec.inline(),
List.of(),
null,
EditorConfig.onUpload()));
}
private void teamPolicy(String name, Long teamId, String... sourceIds) {
@@ -18,6 +18,7 @@ import org.mockito.ArgumentCaptor;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import stirling.software.proprietary.policy.model.EditorConfig;
import stirling.software.proprietary.policy.model.OutputSpec;
import stirling.software.proprietary.policy.model.PipelineInput;
import stirling.software.proprietary.policy.model.PipelineStep;
@@ -113,6 +114,129 @@ class JpaPolicyStoreTest {
upgraded.inputs());
}
/**
* The regression this guards: before the editor lift, a blob written by the pre-{@code editor}
* seeder deserialized straight onto {@link EditorConfig#disabled()}, silently taking every
* upgraded install's Classification policy off the editor.
*
* <p>The {@code inputs} variant is the important one - {@link
* JpaPolicyStore#upgradeLegacyShape} returns early on it, so a lift living inside that method
* would miss exactly the rows written between the trigger migration and this field.
*/
@Test
void getLiftsALegacyEditorSourceOntoEditorConfigWhenInputsArePresent() {
Policy lifted = readLegacy(legacyJson("\"inputs\":[],", "\"sources\":[\"editor\"],"));
assertEquals(EditorConfig.onUpload(), lifted.editor());
assertEquals(Optional.of("upload"), lifted.editorRunOn());
}
@Test
void getLiftsALegacyEditorSourceOnThePreInputsShapeToo() {
// Oldest shape: policy-level trigger + sourceIds, so both migrations have to compose.
Policy lifted =
readLegacy(
legacyJson(
"\"trigger\":{\"type\":\"schedule\",\"options\":{}},"
+ "\"sourceIds\":[\"s1\"],",
"\"sources\":[\"editor\"],"));
assertEquals(EditorConfig.onUpload(), lifted.editor());
assertEquals(
List.of(new PipelineInput("s1", new TriggerConfig("schedule", Map.of()))),
lifted.inputs());
}
@Test
void getTreatsAnUnnarrowedCataloguePolicyAsEditorRun() {
// Empty and absent both meant "nobody narrowed it", which the editor read as its own.
assertTrue(readLegacy(legacyJson("\"inputs\":[],", "\"sources\":[],")).editor().allowed());
assertTrue(readLegacy(legacyJson("\"inputs\":[],", "")).editor().allowed());
}
@Test
void getLeavesACataloguePolicyScopedElsewhereOffTheEditor() {
Policy lifted = readLegacy(legacyJson("\"inputs\":[],", "\"sources\":[\"sharepoint\"],"));
assertFalse(lifted.editor().allowed());
assertEquals(Optional.empty(), lifted.editorRunOn());
}
@Test
void getLeavesASourcelessBuilderPipelineOffTheEditor() {
// No categoryId: a pipeline built on the Pipelines page, which never reached the editor.
String json =
"{\"id\":\"p1\",\"name\":\"legacy\",\"enabled\":true,\"inputs\":[],"
+ "\"steps\":[],\"output\":{\"type\":\"inline\",\"options\":{}}}";
assertFalse(readLegacy(json).editor().allowed());
}
@Test
void getKeepsTheCategoryDefaultMomentWhenNoRunOnWasStored() {
// Security enforced on export before runOn was persisted (frontend runOn.ts
// DEFAULT_RUN_ON).
String json =
"{\"id\":\"p1\",\"name\":\"legacy\",\"enabled\":true,\"inputs\":[],"
+ "\"steps\":[],\"output\":{\"type\":\"inline\",\"options\":{"
+ "\"categoryId\":\"security\",\"sources\":[\"editor\"]}}}";
assertEquals(EditorConfig.onExport(), readLegacy(json).editor());
}
@Test
void getNeverOverridesAnExplicitlyStoredEditorBlock() {
// A deliberate opt-out survives, so the lift stays safe to leave in permanently.
String json =
"{\"id\":\"p1\",\"name\":\"legacy\",\"enabled\":true,\"inputs\":[],"
+ "\"steps\":[],\"editor\":{\"allowed\":false,\"runOn\":\"upload\"},"
+ "\"output\":{\"type\":\"inline\",\"options\":{"
+ "\"categoryId\":\"classification\",\"sources\":[\"editor\"]}}}";
assertFalse(readLegacy(json).editor().allowed());
}
/**
* Pins the wire shape the stubbed Playwright spec hardcodes: the derived block is additive, so
* a real response carries it alongside the untouched legacy options bag.
*/
@Test
void getLeavesTheLegacyOptionsBagIntactSoTheResponseCarriesBoth() {
Policy lifted = readLegacy(legacyJson("\"inputs\":[],", "\"sources\":[\"editor\"],"));
assertEquals(List.of("editor"), lifted.output().options().get("sources"));
String wire = objectMapper.writeValueAsString(lifted);
assertTrue(
wire.contains("\"editor\":{\"allowed\":true,\"runOn\":\"upload\"}"),
"expected the derived editor block on the wire, got: " + wire);
}
/**
* The blob main's DefaultClassificationPolicySeeder wrote, with the shape bits parameterised.
*/
private static String legacyJson(String shapeFields, String sourcesField) {
return "{\"id\":\"p1\",\"name\":\"Classification Policy\",\"owner\":\"system\","
+ "\"enabled\":true,"
+ shapeFields
+ "\"steps\":[{\"operation\":\"/api/v1/ai/tools/classify-and-label\","
+ "\"parameters\":{}}],"
+ "\"output\":{\"type\":\"inline\",\"options\":{"
+ "\"categoryId\":\"classification\",\"runOn\":\"upload\","
+ "\"mode\":\"new_version\","
+ sourcesField
+ "\"scopeTypes\":[],\"reviewerEmail\":\"\"}},\"teamId\":1}";
}
private Policy readLegacy(String policyJson) {
PolicyEntity entity = new PolicyEntity();
entity.setId("p1");
entity.setName("legacy");
entity.setEnabled(true);
entity.setPolicyJson(policyJson);
when(repository.findById("p1")).thenReturn(Optional.of(entity));
return store.get("p1").orElseThrow();
}
@Test
void saveDenormalizesTeamIdForScopedQueries() {
store.save(
@@ -0,0 +1,340 @@
package stirling.software.proprietary.storage.controller;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyBoolean;
import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.ArgumentMatchers.eq;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.never;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.verifyNoInteractions;
import static org.mockito.Mockito.when;
import java.net.URI;
import java.nio.file.Path;
import java.time.Duration;
import java.util.List;
import java.util.Optional;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.junit.jupiter.api.io.TempDir;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import org.mockito.junit.jupiter.MockitoSettings;
import org.mockito.quality.Strictness;
import org.springframework.core.io.ByteArrayResource;
import org.springframework.core.io.Resource;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.security.authentication.UsernamePasswordAuthenticationToken;
import org.springframework.security.core.Authentication;
import org.springframework.web.server.ResponseStatusException;
import stirling.software.common.cluster.inprocess.LocalDiskFileStore;
import stirling.software.proprietary.policy.engine.PolicyEngine;
import stirling.software.proprietary.policy.store.PolicyStore;
import stirling.software.proprietary.security.model.User;
import stirling.software.proprietary.service.AuditService;
import stirling.software.proprietary.storage.egress.ShareEgressDecision;
import stirling.software.proprietary.storage.egress.ShareEgressProcessor;
import stirling.software.proprietary.storage.model.FileShare;
import stirling.software.proprietary.storage.model.ShareAccessRole;
import stirling.software.proprietary.storage.model.StoredFile;
import stirling.software.proprietary.storage.provider.StorageProvider;
import stirling.software.proprietary.storage.repository.FileShareRepository;
import stirling.software.proprietary.storage.service.FileStorageService;
/** The delivery half: what the download endpoints do once a policy has decided. */
@ExtendWith(MockitoExtension.class)
@MockitoSettings(strictness = Strictness.LENIENT)
class FileStorageControllerEgressTest {
private static final String TOKEN = "tok";
private static final String STORAGE_KEY = "11/abc-doc.pdf";
private static final byte[] PNG = {(byte) 0x89, 'P', 'N', 'G'};
@Mock private FileStorageService fileStorageService;
@Mock private StorageProvider storageProvider;
@Mock private ShareEgressProcessor shareEgressProcessor;
@Mock private AuditService auditService;
private FileStorageController controller;
private StoredFile file;
private FileShare share;
private Authentication authentication;
@BeforeEach
void setUp() {
controller =
new FileStorageController(
fileStorageService, storageProvider, shareEgressProcessor, auditService);
file = storedFile();
share = new FileShare();
share.setFile(file);
share.setShareToken(TOKEN);
share.setAccessRole(ShareAccessRole.VIEWER);
authentication = new UsernamePasswordAuthenticationToken(recipient(), "n/a", List.of());
when(fileStorageService.getShareByToken(TOKEN)).thenReturn(share);
when(fileStorageService.canAccessShareLink(share, authentication)).thenReturn(true);
when(fileStorageService.loadFile(file)).thenReturn(new ByteArrayResource(new byte[] {1}));
}
@Test
void aViewOnlyPolicyNeverServesAnAttachment() {
when(fileStorageService.decideDelivery(eq(share), any())).thenReturn(viewOnly());
when(shareEgressProcessor.resolve(eq(share), any(), any(), any()))
.thenReturn(new ByteArrayResource(new byte[] {7}));
// Asking for an attachment is a client hint; the policy decides the disposition.
ResponseEntity<Resource> response =
controller.downloadShareLink(TOKEN, authentication, false);
assertThat(response.getStatusCode()).isEqualTo(HttpStatus.OK);
assertThat(response.getHeaders().getContentDisposition().isInline()).isTrue();
}
@Test
void aViewOnlyDeliveryIsProcessedWhicheverDispositionIsAskedFor() {
ShareEgressDecision decision = viewOnly();
when(fileStorageService.decideDelivery(eq(share), any())).thenReturn(decision);
when(shareEgressProcessor.resolve(eq(share), any(), any(), eq(decision)))
.thenReturn(new ByteArrayResource(new byte[] {7}));
// The bypass this closes: ?inline=true used to skip the gate and hand back the original.
controller.downloadShareLink(TOKEN, authentication, true);
verify(shareEgressProcessor).resolve(eq(share), any(), any(), eq(decision));
}
@Test
void aRefusedDeliveryIsNotRecordedAsAnAccess() {
when(fileStorageService.decideDelivery(eq(share), any())).thenReturn(blocked());
assertThatThrownBy(() -> controller.downloadShareLink(TOKEN, authentication, false))
.isInstanceOf(ResponseStatusException.class);
// Otherwise the owner's access log would show a view that never happened.
verify(fileStorageService, never()).recordShareAccess(any(), any(), anyBoolean());
}
@Test
void aViewOnlyAccessIsLoggedAsAViewNotADownload() {
when(fileStorageService.decideDelivery(eq(share), any())).thenReturn(viewOnly());
when(shareEgressProcessor.resolve(eq(share), any(), any(), any()))
.thenReturn(new ByteArrayResource(new byte[] {7}));
controller.downloadShareLink(TOKEN, authentication, false);
verify(fileStorageService).recordShareAccess(share, authentication, true);
}
@Test
void aGovernedDeliveryNeverRedirectsToASignedUrl() throws Exception {
when(fileStorageService.decideDelivery(eq(share), any())).thenReturn(viewOnly());
when(shareEgressProcessor.resolve(eq(share), any(), any(), any()))
.thenReturn(new ByteArrayResource(new byte[] {7}));
when(storageProvider.signedDownloadUrl(
eq(STORAGE_KEY), any(Duration.class), anyBoolean(), anyString()))
.thenReturn(Optional.of(URI.create("https://bucket.example/signed")));
ResponseEntity<Resource> response =
controller.downloadShareLink(TOKEN, authentication, true);
// A signed URL points straight at the stored object, so it would hand over the unprocessed
// document and ignore the view-only disposition.
assertThat(response.getStatusCode()).isEqualTo(HttpStatus.OK);
assertThat(response.getBody()).isNotNull();
}
@Test
void aTransformingPolicyServesTheProcessedCopy() {
ShareEgressDecision decision = transforming();
byte[] processed = new byte[] {1, 2, 3, 4, 5};
when(fileStorageService.decideDelivery(eq(share), any())).thenReturn(decision);
when(shareEgressProcessor.resolve(eq(share), any(), any(), eq(decision)))
.thenReturn(new ByteArrayResource(processed));
ResponseEntity<Resource> response =
controller.downloadShareLink(TOKEN, authentication, false);
assertThat(response.getStatusCode()).isEqualTo(HttpStatus.OK);
assertThat(response.getBody()).isInstanceOf(ByteArrayResource.class);
// The header must describe the copy actually served, not the stored original.
assertThat(response.getHeaders().getContentLength()).isEqualTo(processed.length);
}
@Test
void anUngovernedDeliveryIsLeftExactlyAsItWas() {
when(fileStorageService.decideDelivery(eq(share), any()))
.thenReturn(ShareEgressDecision.unrestricted(ShareAccessRole.VIEWER));
ResponseEntity<Resource> response =
controller.downloadShareLink(TOKEN, authentication, false);
assertThat(response.getStatusCode()).isEqualTo(HttpStatus.OK);
verifyNoInteractions(shareEgressProcessor);
}
@Test
void anOwnerDownloadingTheirOwnFileIsNotEgress() {
User owner = ownerUser();
when(fileStorageService.requireAuthenticatedUser()).thenReturn(owner);
when(fileStorageService.getAccessibleFile(owner, 77L)).thenReturn(file);
when(fileStorageService.findUserShare(owner, file)).thenReturn(Optional.empty());
controller.downloadFile(77L, false);
verify(fileStorageService, never()).decideDelivery(any(), any());
verifyNoInteractions(shareEgressProcessor);
}
@Test
void aRecipientDownloadingAFileSharedWithThemIsEgress() {
User user = recipient();
ShareEgressDecision decision = transforming();
FileShare userShare = new FileShare();
userShare.setFile(file);
userShare.setSharedWithUser(user);
userShare.setAccessRole(ShareAccessRole.VIEWER);
when(fileStorageService.requireAuthenticatedUser()).thenReturn(user);
when(fileStorageService.getAccessibleFile(user, 77L)).thenReturn(file);
when(fileStorageService.findUserShare(user, file)).thenReturn(Optional.of(userShare));
when(fileStorageService.decideDelivery(eq(userShare), any())).thenReturn(decision);
when(shareEgressProcessor.resolve(eq(userShare), any(), any(), eq(decision)))
.thenReturn(new ByteArrayResource(new byte[] {9}));
ResponseEntity<Resource> response = controller.downloadFile(77L, false);
assertThat(response.getStatusCode()).isEqualTo(HttpStatus.OK);
verify(shareEgressProcessor).resolve(eq(userShare), any(), any(), eq(decision));
// Direct downloads are not share-link accesses, so nothing is logged against a token.
verify(fileStorageService, never()).recordShareAccess(any(), any(), anyBoolean());
}
@Test
void aNonPdfSharedWithAnExternalRecipientIsStillDelivered(@TempDir Path tempDir) {
// externalRecipients=restrict makes every external recipient view-only, and the rasterising
// pass only understands PDFs; running it here would refuse the share outright.
StoredFile image = storedFile();
image.setOriginalFilename("photo.png");
image.setContentType("image/png");
share.setFile(image);
PolicyEngine policyEngine = mock(PolicyEngine.class);
FileStorageController withRealProcessor =
new FileStorageController(
fileStorageService,
storageProvider,
new ShareEgressProcessor(
mock(PolicyStore.class),
policyEngine,
new LocalDiskFileStore(tempDir.toString()),
mock(FileShareRepository.class)),
auditService);
when(fileStorageService.loadFile(image)).thenReturn(new ByteArrayResource(PNG));
when(fileStorageService.decideDelivery(eq(share), any())).thenReturn(viewOnly());
ResponseEntity<Resource> response =
withRealProcessor.downloadShareLink(TOKEN, authentication, false);
assertThat(response.getStatusCode()).isEqualTo(HttpStatus.OK);
assertThat(response.getHeaders().getContentDisposition().isInline()).isTrue();
verifyNoInteractions(policyEngine);
}
@Test
void twoDifferentRecipientsOnOneLinkAreBothServed() {
ShareEgressDecision decision = transforming();
Authentication second =
new UsernamePasswordAuthenticationToken(secondRecipient(), "n/a", List.of());
when(fileStorageService.canAccessShareLink(share, second)).thenReturn(true);
when(fileStorageService.decideDelivery(eq(share), any())).thenReturn(decision);
when(shareEgressProcessor.resolve(eq(share), any(), any(), eq(decision)))
.thenReturn(new ByteArrayResource(new byte[] {9}));
assertThat(controller.downloadShareLink(TOKEN, authentication, false).getStatusCode())
.isEqualTo(HttpStatus.OK);
// The processed copy is cached against the share, so the second identity reads a copy the
// first one's run produced.
assertThat(controller.downloadShareLink(TOKEN, second, false).getStatusCode())
.isEqualTo(HttpStatus.OK);
}
private static User secondRecipient() {
User user = new User();
user.setId(33L);
user.setUsername("carol@partner.com");
return user;
}
private static ShareEgressDecision viewOnly() {
return new ShareEgressDecision(
true,
null,
ShareAccessRole.VIEWER,
1,
true,
true,
"policy-1",
"Sharing Policy",
List.of(),
null);
}
private static ShareEgressDecision blocked() {
return new ShareEgressDecision(
false,
"This document may not be shared outside your organisation",
null,
null,
false,
true,
"policy-1",
"Sharing Policy",
List.of(),
null);
}
private static ShareEgressDecision transforming() {
return new ShareEgressDecision(
true,
null,
ShareAccessRole.VIEWER,
null,
false,
false,
"policy-1",
"Sharing Policy",
List.of("policy-1"),
"fingerprint");
}
private static User ownerUser() {
User user = new User();
user.setId(11L);
user.setUsername("alice@example.com");
return user;
}
private static User recipient() {
User user = new User();
user.setId(22L);
user.setUsername("bob@partner.com");
return user;
}
private static StoredFile storedFile() {
StoredFile file = new StoredFile();
file.setId(77L);
file.setOwner(ownerUser());
file.setOriginalFilename("doc.pdf");
file.setContentType("application/pdf");
file.setSizeBytes(123L);
file.setStorageKey(STORAGE_KEY);
return file;
}
}
@@ -32,6 +32,8 @@ import org.springframework.web.server.ResponseStatusException;
import stirling.software.proprietary.security.model.User;
import stirling.software.proprietary.service.AuditService;
import stirling.software.proprietary.storage.egress.ShareEgressDecision;
import stirling.software.proprietary.storage.egress.ShareEgressProcessor;
import stirling.software.proprietary.storage.model.FileShare;
import stirling.software.proprietary.storage.model.ShareAccessRole;
import stirling.software.proprietary.storage.model.StoredFile;
@@ -49,13 +51,16 @@ class FileStorageControllerMoreTest {
@Mock private FileStorageService fileStorageService;
@Mock private StorageProvider storageProvider;
@Mock private ShareEgressProcessor shareEgressProcessor;
@Mock private AuditService auditService;
private FileStorageController controller;
@BeforeEach
void setUp() {
controller = new FileStorageController(fileStorageService, storageProvider, auditService);
controller =
new FileStorageController(
fileStorageService, storageProvider, shareEgressProcessor, auditService);
}
private User user() {
@@ -335,6 +340,9 @@ class FileStorageControllerMoreTest {
Resource resource = new ByteArrayResource(new byte[] {9});
when(fileStorageService.getShareByToken("tok")).thenReturn(share);
when(fileStorageService.canAccessShareLink(share, authentication)).thenReturn(true);
// No sharing policy governs this file, so delivery is untouched.
when(fileStorageService.decideDelivery(eq(share), any()))
.thenReturn(ShareEgressDecision.unrestricted(ShareAccessRole.VIEWER));
when(storageProvider.signedDownloadUrl(
eq("11/abc-doc.pdf"), any(Duration.class), anyBoolean(), anyString()))
.thenReturn(Optional.empty());
@@ -36,6 +36,7 @@ import org.springframework.test.web.servlet.setup.MockMvcBuilders;
import stirling.software.proprietary.audit.AuditEventType;
import stirling.software.proprietary.security.model.User;
import stirling.software.proprietary.service.AuditService;
import stirling.software.proprietary.storage.egress.ShareEgressProcessor;
import stirling.software.proprietary.storage.model.StoredFile;
import stirling.software.proprietary.storage.provider.StorageProvider;
import stirling.software.proprietary.storage.service.FileStorageService;
@@ -48,6 +49,7 @@ class FileStorageControllerTest {
@Mock private FileStorageService fileStorageService;
@Mock private StorageProvider storageProvider;
@Mock private ShareEgressProcessor shareEgressProcessor;
@Mock private AuditService auditService;
private MockMvc mockMvc;
@@ -55,7 +57,8 @@ class FileStorageControllerTest {
@BeforeEach
void setUp() {
FileStorageController controller =
new FileStorageController(fileStorageService, storageProvider, auditService);
new FileStorageController(
fileStorageService, storageProvider, shareEgressProcessor, auditService);
mockMvc = MockMvcBuilders.standaloneSetup(controller).build();
}
@@ -0,0 +1,104 @@
package stirling.software.proprietary.storage.egress;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
import java.util.List;
import java.util.Map;
import java.util.Set;
import org.junit.jupiter.api.Test;
import stirling.software.proprietary.policy.model.OutputSpec;
import stirling.software.proprietary.policy.model.PipelineStep;
import stirling.software.proprietary.policy.model.Policy;
import stirling.software.proprietary.storage.model.ShareAccessRole;
/** Reading a policy's untyped options bag as a typed rule, and still coming out usable. */
class EgressRuleTest {
@Test
void readsTheSettingsThePortalPersists() {
EgressRule rule =
EgressRule.from(
sharingPolicy(
Map.of(
"categoryId", "sharing",
"sources", List.of("shareLink", "emailShare"),
"fieldValues",
Map.of(
"defaultAccess", "restricted",
"externalRecipients", "block",
"internalDomains",
List.of("@example.com", "*.co.uk"),
"linkExpiry", "threeDays",
"downloads", "viewOnly"))));
assertEquals(Set.of(ShareChannel.SHARE_LINK, ShareChannel.EMAIL_SHARE), rule.channels());
assertEquals(ShareAccessRole.VIEWER, rule.maxRole());
assertEquals(EgressRule.ExternalRecipients.BLOCK, rule.externalRecipients());
assertEquals(Set.of("example.com", "co.uk"), rule.internalDomains());
assertEquals(3, rule.maxLinkDays());
assertTrue(rule.viewOnly());
}
@Test
void anEmptyBagIsAnUnnarrowedRuleThatRestrictsExternalRecipients() {
EgressRule rule = EgressRule.from(sharingPolicy(Map.of()));
assertTrue(rule.channels().isEmpty());
assertNull(rule.maxRole());
assertNull(rule.maxLinkDays());
assertFalse(rule.viewOnly());
// The safe default when the setting is absent, not "allow".
assertEquals(EgressRule.ExternalRecipients.RESTRICT, rule.externalRecipients());
}
@Test
void unrecognisedValuesFallBackToNoCeilingRatherThanThrowing() {
EgressRule rule =
EgressRule.from(
sharingPolicy(
Map.of(
"sources", List.of("editor", "notAChannel"),
"fieldValues",
Map.of(
"defaultAccess", "inherit",
"linkExpiry", "inherit"))));
assertTrue(rule.channels().isEmpty(), "unknown channel ids are dropped");
assertNull(rule.maxRole());
assertNull(rule.maxLinkDays());
}
@Test
void anUnnarrowedRuleCoversEveryChannel() {
EgressRule rule = EgressRule.from(sharingPolicy(Map.of()));
assertTrue(rule.covers(ShareChannel.USER_SHARE));
assertTrue(rule.covers(ShareChannel.SHARE_LINK));
}
@Test
void aNarrowedRuleCoversOnlyItsChannels() {
EgressRule rule = EgressRule.from(sharingPolicy(Map.of("sources", List.of("shareLink"))));
assertTrue(rule.covers(ShareChannel.SHARE_LINK));
assertFalse(rule.covers(ShareChannel.USER_SHARE));
}
private static Policy sharingPolicy(Map<String, Object> options) {
return new Policy(
"p1",
"Sharing Policy",
"owner@example.com",
true,
List.of(),
List.of(new PipelineStep("/api/v1/security/add-watermark", Map.of(), Map.of())),
new OutputSpec("inline", options),
List.of(),
7L);
}
}
@@ -0,0 +1,411 @@
package stirling.software.proprietary.storage.egress;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
import java.time.LocalDateTime;
import java.util.List;
import java.util.Map;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import stirling.software.proprietary.model.Team;
import stirling.software.proprietary.policy.model.OutputSpec;
import stirling.software.proprietary.policy.model.PipelineStep;
import stirling.software.proprietary.policy.model.Policy;
import stirling.software.proprietary.policy.store.InProcessPolicyStore;
import stirling.software.proprietary.policy.store.PolicyStore;
import stirling.software.proprietary.security.model.User;
import stirling.software.proprietary.storage.model.FileShare;
import stirling.software.proprietary.storage.model.ShareAccessRole;
import stirling.software.proprietary.storage.model.StoredFile;
import tools.jackson.databind.json.JsonMapper;
/** The egress decision: who a share may go to, on what terms, and what must be processed. */
class ShareEgressPolicyServiceTest {
private static final Long TEAM = 7L;
private static final String WATERMARK = "/api/v1/security/add-watermark";
private PolicyStore policyStore;
private ShareEgressPolicyService service;
private User owner;
private StoredFile file;
@BeforeEach
void setUp() {
policyStore = new InProcessPolicyStore();
service = new ShareEgressPolicyService(policyStore, JsonMapper.builder().build());
owner = user("alice@example.com", TEAM);
file = storedFile(owner);
}
@Test
void withNoSharingPolicyNothingChanges() {
ShareEgressDecision decision = grant("bob@partner.com", ShareAccessRole.EDITOR);
assertTrue(decision.allowed());
assertEquals(ShareAccessRole.EDITOR, decision.role());
assertNull(decision.maxLinkDays());
assertFalse(decision.viewOnly());
assertFalse(decision.transforms());
assertNull(decision.policyId());
}
@Test
void anotherTeamsSharingPolicyDoesNotGovernThisOwner() {
savePolicy(sharing(Map.of("defaultAccess", "restricted"), List.of(), 99L));
assertEquals(
ShareAccessRole.EDITOR, grant("bob@example.com", ShareAccessRole.EDITOR).role());
}
@Test
void aPolicyFromAnotherCategoryDoesNotGovernEgress() {
policyStore.save(
new Policy(
null,
"Security Policy",
owner.getUsername(),
true,
List.of(),
List.of(new PipelineStep(WATERMARK, Map.of(), Map.of())),
new OutputSpec(
"inline",
Map.of(
"categoryId",
"security",
"fieldValues",
Map.of("externalRecipients", "block"))),
TEAM));
assertTrue(grant("bob@partner.com", ShareAccessRole.EDITOR).allowed());
}
@Test
void adisabledPolicyDoesNotGovernAnything() {
Policy policy = sharing(Map.of("externalRecipients", "block"), List.of(), TEAM);
policyStore.save(
new Policy(
policy.id(),
policy.name(),
policy.owner(),
false,
policy.inputs(),
policy.steps(),
policy.output(),
policy.outputIds(),
policy.teamId()));
assertTrue(grant("bob@partner.com", ShareAccessRole.EDITOR).allowed());
}
@Test
void defaultAccessCapsTheRoleAShareMayGrant() {
savePolicy(
sharing(
Map.of("defaultAccess", "restricted", "externalRecipients", "allow"),
List.of(),
TEAM));
assertEquals(
ShareAccessRole.VIEWER, grant("bob@example.com", ShareAccessRole.EDITOR).role());
}
@Test
void theCapOnlyTightens_itNeverPromotesAWeakerRequest() {
savePolicy(
sharing(
Map.of("defaultAccess", "editor", "externalRecipients", "allow"),
List.of(),
TEAM));
assertEquals(
ShareAccessRole.VIEWER, grant("bob@example.com", ShareAccessRole.VIEWER).role());
}
@Test
void blockingExternalRecipientsRefusesTheShareAndNamesThePolicy() {
savePolicy(sharing(Map.of("externalRecipients", "block"), List.of(), TEAM));
ShareEgressDecision decision = grant("bob@partner.com", ShareAccessRole.EDITOR);
assertFalse(decision.allowed());
assertTrue(decision.external());
assertNotNull(decision.reason());
assertEquals("Sharing Policy", decision.policyName());
}
@Test
void aRecipientOnTheOwnersOwnDomainIsInternalWhenNoDomainsAreConfigured() {
savePolicy(sharing(Map.of("externalRecipients", "block"), List.of(), TEAM));
ShareEgressDecision decision = grant("bob@example.com", ShareAccessRole.EDITOR);
assertTrue(decision.allowed());
assertFalse(decision.external());
}
@Test
void configuredInternalDomainsWinOverTheOwnersOwn() {
savePolicy(
sharing(
Map.of(
"externalRecipients",
"block",
"internalDomains",
List.of("partner.com")),
List.of(),
TEAM));
// partner.com is now inside; the owner's own example.com is not.
assertTrue(grant("bob@partner.com", ShareAccessRole.EDITOR).allowed());
assertFalse(grant("bob@example.com", ShareAccessRole.EDITOR).allowed());
}
@Test
void aBareUsernameWithNoDomainIsTreatedAsInternal() {
savePolicy(sharing(Map.of("externalRecipients", "block"), List.of(), TEAM));
// Only a registered local user can hold one, so there is nothing external about it.
assertTrue(grant("bob", ShareAccessRole.EDITOR).allowed());
}
@Test
void restrictingExternalRecipientsGivesReadOnlyViewOnlyAccessOnAOneDayLink() {
savePolicy(sharing(Map.of("externalRecipients", "restrict"), List.of(), TEAM));
ShareEgressDecision decision = grant("bob@partner.com", ShareAccessRole.EDITOR);
assertTrue(decision.allowed());
assertTrue(decision.external());
assertEquals(ShareAccessRole.VIEWER, decision.role());
assertTrue(decision.viewOnly());
assertEquals(1, decision.maxLinkDays());
}
@Test
void restrictLeavesInternalRecipientsAlone() {
savePolicy(sharing(Map.of("externalRecipients", "restrict"), List.of(), TEAM));
ShareEgressDecision decision = grant("bob@example.com", ShareAccessRole.EDITOR);
assertFalse(decision.viewOnly());
assertNull(decision.maxLinkDays());
}
@Test
void aPolicyNarrowedToOneChannelIgnoresTheOthers() {
savePolicy(sharing(Map.of("externalRecipients", "block"), List.of("shareLink"), TEAM));
assertTrue(
service.evaluateGrant(
ShareChannel.USER_SHARE,
file,
owner,
"bob@partner.com",
ShareAccessRole.EDITOR)
.allowed());
assertFalse(
service.evaluateGrant(
ShareChannel.SHARE_LINK,
file,
owner,
"bob@partner.com",
ShareAccessRole.EDITOR)
.allowed());
}
@Test
void severalPoliciesComposeToTheMostRestrictiveOutcome() {
savePolicy(
sharing(
Map.of(
"defaultAccess", "commenter",
"externalRecipients", "allow",
"linkExpiry", "thirtyDays"),
List.of(),
TEAM));
savePolicy(
sharing(
Map.of(
"defaultAccess", "inherit",
"externalRecipients", "allow",
"linkExpiry", "threeDays",
"downloads", "viewOnly"),
List.of(),
TEAM));
ShareEgressDecision decision = grant("bob@example.com", ShareAccessRole.EDITOR);
assertEquals(ShareAccessRole.COMMENTER, decision.role());
assertEquals(3, decision.maxLinkDays());
assertTrue(decision.viewOnly());
}
@Test
void aPolicyWithStepsMarksTheCopyForProcessing() {
Policy saved = savePolicy(sharing(Map.of(), List.of(), TEAM));
ShareEgressDecision decision = grant("bob@example.com", ShareAccessRole.VIEWER);
assertTrue(decision.transforms());
assertEquals(List.of(saved.id()), decision.transformPolicyIds());
assertNotNull(decision.transformFingerprint());
assertTrue(decision.requiresManagedDelivery());
}
@Test
void viewOnlyWithNoStepsStillProcessesTheCopyThatLeaves() {
policyStore.save(stepless(Map.of("downloads", "viewOnly")));
ShareEgressDecision decision = grant("bob@example.com", ShareAccessRole.VIEWER);
assertTrue(decision.viewOnly());
assertFalse(decision.transforms());
// The rasterising pass is what makes view-only mean anything, so it caches like a chain.
assertNotNull(decision.transformFingerprint());
assertTrue(decision.requiresManagedDelivery());
}
@Test
void aPolicyWithNoStepsAndNoViewOnlyGovernsAccessWithoutTouchingTheBytes() {
policyStore.save(stepless(Map.of("defaultAccess", "restricted")));
ShareEgressDecision decision = grant("bob@example.com", ShareAccessRole.VIEWER);
assertFalse(decision.viewOnly());
assertFalse(decision.transforms());
assertNull(decision.transformFingerprint());
assertFalse(decision.requiresManagedDelivery());
}
@Test
void anEmailShareIsStillJudgedAsOneWhenTheLinkItMintedIsOpened() {
savePolicy(sharing(Map.of("externalRecipients", "block"), List.of("emailShare"), TEAM));
FileShare share = new FileShare();
share.setFile(file);
share.setShareToken("token");
share.setAccessRole(ShareAccessRole.VIEWER);
share.setEgressChannel(ShareChannel.EMAIL_SHARE);
// Without the stamped channel this reads as a plain link share and the policy never bites.
assertFalse(service.evaluateDelivery(share, user("bob@partner.com", 99L)).allowed());
}
@Test
void aLinkShareIsNotGovernedByAnEmailOnlyPolicy() {
savePolicy(sharing(Map.of("externalRecipients", "block"), List.of("emailShare"), TEAM));
FileShare share = new FileShare();
share.setFile(file);
share.setShareToken("token");
share.setAccessRole(ShareAccessRole.VIEWER);
share.setEgressChannel(ShareChannel.SHARE_LINK);
assertTrue(service.evaluateDelivery(share, user("bob@partner.com", 99L)).allowed());
}
@Test
void aUserShareIsJudgedOnTheUserShareChannel() {
savePolicy(sharing(Map.of("externalRecipients", "block"), List.of("userShare"), TEAM));
FileShare share = new FileShare();
share.setFile(file);
share.setSharedWithUser(user("bob@partner.com", 99L));
share.setAccessRole(ShareAccessRole.VIEWER);
share.setEgressChannel(ShareChannel.USER_SHARE);
assertFalse(service.evaluateDelivery(share, user("bob@partner.com", 99L)).allowed());
}
@Test
void theFingerprintChangesWhenTheDocumentIsReplaced() {
savePolicy(sharing(Map.of(), List.of(), TEAM));
String before = grant("bob@example.com", ShareAccessRole.VIEWER).transformFingerprint();
file.setUpdatedAt(file.getUpdatedAt().plusMinutes(1));
file.setSizeBytes(999L);
assertFalse(
before.equals(
grant("bob@example.com", ShareAccessRole.VIEWER).transformFingerprint()));
}
@Test
void deliveryOfALinkShareIsJudgedAgainstWhoeverOpensIt() {
savePolicy(sharing(Map.of("externalRecipients", "block"), List.of(), TEAM));
FileShare share = new FileShare();
share.setFile(file);
share.setShareToken("token");
share.setAccessRole(ShareAccessRole.VIEWER);
assertFalse(service.evaluateDelivery(share, user("bob@partner.com", 99L)).allowed());
assertTrue(service.evaluateDelivery(share, user("bob@example.com", TEAM)).allowed());
}
private ShareEgressDecision grant(String recipient, ShareAccessRole role) {
return service.evaluateGrant(ShareChannel.USER_SHARE, file, owner, recipient, role);
}
private Policy savePolicy(Policy policy) {
return policyStore.save(policy);
}
/** A Sharing policy that governs access only: no tool chain to run over the copy. */
private Policy stepless(Map<String, Object> fieldValues) {
return new Policy(
null,
"Sharing Policy",
owner.getUsername(),
true,
List.of(),
List.of(),
new OutputSpec(
"inline", Map.of("categoryId", "sharing", "fieldValues", fieldValues)),
TEAM);
}
private Policy sharing(Map<String, Object> fieldValues, List<String> channels, Long teamId) {
return new Policy(
null,
"Sharing Policy",
owner.getUsername(),
true,
List.of(),
List.of(new PipelineStep(WATERMARK, Map.of(), Map.of())),
new OutputSpec(
"inline",
Map.of(
"categoryId", "sharing",
"sources", channels,
"fieldValues", fieldValues)),
List.of(),
teamId);
}
private static User user(String username, Long teamId) {
User user = new User();
user.setId(Math.abs((long) username.hashCode()));
user.setUsername(username);
Team team = new Team();
team.setId(teamId);
team.setName("Team " + teamId);
user.setTeam(team);
return user;
}
private static StoredFile storedFile(User owner) {
StoredFile file = new StoredFile();
file.setId(1L);
file.setOwner(owner);
file.setOriginalFilename("contract.pdf");
file.setSizeBytes(1024L);
file.setStorageKey("key");
file.setUpdatedAt(LocalDateTime.of(2026, 7, 30, 12, 0));
return file;
}
}
@@ -0,0 +1,358 @@
package stirling.software.proprietary.storage.egress;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.verifyNoInteractions;
import static org.mockito.Mockito.when;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.nio.file.Path;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.concurrent.CompletableFuture;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.junit.jupiter.api.io.TempDir;
import org.mockito.ArgumentCaptor;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import org.mockito.junit.jupiter.MockitoSettings;
import org.mockito.quality.Strictness;
import org.springframework.core.io.ByteArrayResource;
import org.springframework.core.io.Resource;
import stirling.software.common.cluster.FileStore;
import stirling.software.common.cluster.inprocess.LocalDiskFileStore;
import stirling.software.common.model.job.ResultFile;
import stirling.software.proprietary.policy.engine.PolicyEngine;
import stirling.software.proprietary.policy.engine.PolicyRunHandle;
import stirling.software.proprietary.policy.model.OutputSpec;
import stirling.software.proprietary.policy.model.PipelineDefinition;
import stirling.software.proprietary.policy.model.PipelineStep;
import stirling.software.proprietary.policy.model.Policy;
import stirling.software.proprietary.policy.model.PolicyInputs;
import stirling.software.proprietary.policy.model.PolicyRun;
import stirling.software.proprietary.policy.store.PolicyStore;
import stirling.software.proprietary.storage.model.FileShare;
import stirling.software.proprietary.storage.model.ShareAccessRole;
import stirling.software.proprietary.storage.repository.FileShareRepository;
/**
* The fail-closed path: what a recipient actually receives, and what happens when the policy cannot
* produce it.
*/
@ExtendWith(MockitoExtension.class)
@MockitoSettings(strictness = Strictness.LENIENT)
class ShareEgressProcessorTest {
private static final String POLICY_ID = "policy-1";
private static final String FINGERPRINT = "fingerprint";
private static final String FLATTEN = "/api/v1/misc/flatten";
private static final String PDF_NAME = "doc.pdf";
private static final byte[] ORIGINAL = {1, 1, 1};
private static final byte[] PROCESSED = {2, 2, 2, 2};
@Mock private PolicyStore policyStore;
@Mock private PolicyEngine policyEngine;
@Mock private FileShareRepository fileShareRepository;
private FileStore fileStore;
private ShareEgressProcessor processor;
private FileShare share;
@BeforeEach
void setUp(@TempDir Path tempDir) {
fileStore = new LocalDiskFileStore(tempDir.toString());
processor =
new ShareEgressProcessor(policyStore, policyEngine, fileStore, fileShareRepository);
share = new FileShare();
share.setId(5L);
share.setShareToken("tok");
share.setAccessRole(ShareAccessRole.VIEWER);
when(policyStore.get(POLICY_ID)).thenReturn(Optional.of(watermarkPolicy()));
}
@Test
void anUngovernedDeliveryServesTheStoredOriginal() {
Resource original = original();
Resource served = processor.resolve(share, original, PDF_NAME, unrestricted());
assertThat(served).isSameAs(original);
verifyNoInteractions(policyEngine);
}
@Test
void aPolicyChainProducesTheCopyThatIsServed() throws Exception {
runProduces(PROCESSED);
Resource served = processor.resolve(share, original(), PDF_NAME, transforming());
assertThat(served.getContentAsByteArray()).isEqualTo(PROCESSED);
assertThat(served.getFilename()).isEqualTo("doc.pdf");
}
@Test
void theProcessedCopyIsCachedAgainstTheShare() throws Exception {
String fileId = runProduces(PROCESSED);
processor.resolve(share, original(), PDF_NAME, transforming());
verify(fileShareRepository).save(share);
assertThat(share.getEgressFileId()).isEqualTo(fileId);
assertThat(share.getEgressFingerprint()).isEqualTo(FINGERPRINT);
}
@Test
void aSecondRecipientIsServedTheCachedCopyWithoutRerunningThePolicy() throws Exception {
// Stored owned by whoever's download derived it; a different recipient on the same link
// must still be able to read it (this used to throw SecurityException and 500).
String cached = store(PROCESSED, "recipient-a@partner.com");
share.setEgressFileId(cached);
share.setEgressFingerprint(FINGERPRINT);
Resource served = processor.resolve(share, original(), PDF_NAME, transforming());
assertThat(served.getContentAsByteArray()).isEqualTo(PROCESSED);
verifyNoInteractions(policyEngine);
}
@Test
void aCopyStampedForADifferentDecisionIsRederived() throws Exception {
share.setEgressFileId(store(new byte[] {9}, "recipient-a@partner.com"));
share.setEgressFingerprint("stale");
runProduces(PROCESSED);
Resource served = processor.resolve(share, original(), PDF_NAME, transforming());
assertThat(served.getContentAsByteArray()).isEqualTo(PROCESSED);
}
@Test
void aSweptCacheEntryIsRederived() throws Exception {
share.setEgressFileId("6f1a3f26-0c1a-4a1e-9c2f-2f0f9a5a1b2c");
share.setEgressFingerprint(FINGERPRINT);
runProduces(PROCESSED);
assertThat(
processor
.resolve(share, original(), PDF_NAME, transforming())
.getContentAsByteArray())
.isEqualTo(PROCESSED);
}
@Test
void aViewOnlyDecisionRasterisesEvenWithNoToolChain() throws Exception {
runProduces(PROCESSED);
Resource served = processor.resolve(share, original(), PDF_NAME, viewOnly(List.of()));
assertThat(served.getContentAsByteArray()).isEqualTo(PROCESSED);
assertThat(ranOperations()).containsExactly(FLATTEN);
}
@Test
void aViewOnlyDecisionRasterisesAfterTheToolChain() throws Exception {
runProduces(PROCESSED);
processor.resolve(share, original(), PDF_NAME, viewOnly(List.of(POLICY_ID)));
assertThat(ranOperations()).containsExactly("/api/v1/security/add-watermark", FLATTEN);
}
@Test
void aViewOnlyDeliveryOfANonPdfServesTheStoredBytes() throws Exception {
// The rasteriser only understands PDFs, so running it here would refuse the delivery
// outright. View-only still binds: the caller serves it inline, never as an attachment.
Resource served = processor.resolve(share, original(), "photo.png", viewOnly(List.of()));
assertThat(served.getContentAsByteArray()).isEqualTo(ORIGINAL);
verifyNoInteractions(policyEngine);
assertThat(share.getEgressFileId()).isNull();
}
@Test
void aNonPdfStillRunsTheToolChainTheSharingPolicyConfigured() throws Exception {
runProduces(PROCESSED);
processor.resolve(share, original(), "photo.png", viewOnly(List.of(POLICY_ID)));
// Only the rasterising pass is skipped; whether a tool accepts the type is the chain's own
// business, and a step that refuses it still fails the delivery closed.
assertThat(ranOperations()).containsExactly("/api/v1/security/add-watermark");
}
@Test
void aViewOnlyPayloadOfUnknownTypeIsStillRasterised() throws Exception {
runProduces(PROCESSED);
// Nothing says it is not a PDF, so it fails closed rather than handing over the original.
processor.resolve(share, original(), "scan-no-extension", viewOnly(List.of()));
assertThat(ranOperations()).containsExactly(FLATTEN);
}
@Test
void theToolChainSeesTheDocumentsOwnNameNotTheStoredResources() throws Exception {
runProduces(PROCESSED);
// Providers hand back unnamed resources (database, S3, decrypting), and the engine reads
// the accepted type off the filename.
processor.resolve(share, new ByteArrayResource(ORIGINAL), PDF_NAME, transforming());
assertThat(ranInputNames()).containsExactly(PDF_NAME);
}
@Test
void aFailedRunReleasesNothing() {
PolicyRun run = new PolicyRun("run-1", POLICY_ID, definition(), null, null, null);
run.fail("watermark blew up");
when(policyEngine.runPolicy(any(), any(), any()))
.thenReturn(new PolicyRunHandle("run-1", CompletableFuture.completedFuture(run)));
assertThatThrownBy(() -> processor.resolve(share, original(), PDF_NAME, transforming()))
.isInstanceOf(ShareEgressException.class)
.hasMessageContaining("was not released");
}
@Test
void aRunThatBlowsUpIsCancelledAndReleasesNothing() {
when(policyEngine.runPolicy(any(), any(), any()))
.thenReturn(
new PolicyRunHandle(
"run-1",
CompletableFuture.failedFuture(new IllegalStateException("boom"))));
assertThatThrownBy(() -> processor.resolve(share, original(), PDF_NAME, transforming()))
.isInstanceOf(ShareEgressException.class);
verify(policyEngine).cancel("run-1");
}
@Test
void aPolicyDeletedMidFlightReleasesNothing() {
when(policyStore.get(anyString())).thenReturn(Optional.empty());
assertThatThrownBy(() -> processor.resolve(share, original(), PDF_NAME, transforming()))
.isInstanceOf(ShareEgressException.class)
.hasMessageContaining("no longer available");
verifyNoInteractions(policyEngine);
}
@Test
void aCacheWriteThatFailsDoesNotFailTheDelivery() throws Exception {
runProduces(PROCESSED);
when(fileShareRepository.save(any())).thenThrow(new IllegalStateException("db down"));
assertThat(
processor
.resolve(share, original(), PDF_NAME, transforming())
.getContentAsByteArray())
.isEqualTo(PROCESSED);
}
/** Every operation the engine was asked to run, in order. */
private List<String> ranOperations() {
ArgumentCaptor<Policy> captor = ArgumentCaptor.forClass(Policy.class);
verify(policyEngine, org.mockito.Mockito.atLeastOnce())
.runPolicy(captor.capture(), any(), any());
return captor.getAllValues().stream()
.flatMap(policy -> policy.steps().stream())
.map(PipelineStep::operation)
.toList();
}
/** The filename each run's primary input carried into the engine. */
private List<String> ranInputNames() {
ArgumentCaptor<PolicyInputs> captor = ArgumentCaptor.forClass(PolicyInputs.class);
verify(policyEngine, org.mockito.Mockito.atLeastOnce())
.runPolicy(any(), captor.capture(), any());
return captor.getAllValues().stream()
.flatMap(inputs -> inputs.primary().stream())
.map(Resource::getFilename)
.toList();
}
/** Stubs the engine so every run stores {@code bytes} and completes; returns the last id. */
private String runProduces(byte[] bytes) throws IOException {
String fileId = store(bytes, "recipient-a@partner.com");
when(policyEngine.runPolicy(any(), any(), any()))
.thenAnswer(
invocation -> {
PolicyRun run =
new PolicyRun(
"run-1", POLICY_ID, definition(), null, null, null);
run.complete(List.of(ResultFile.builder().fileId(fileId).build()));
return new PolicyRunHandle(
"run-1", CompletableFuture.completedFuture(run));
});
return fileId;
}
private String store(byte[] bytes, String owner) throws IOException {
return fileStore.store(new ByteArrayInputStream(bytes), "processed.pdf", owner).fileId();
}
private static Resource original() {
return new ByteArrayResource(ORIGINAL) {
@Override
public String getFilename() {
return "doc.pdf";
}
};
}
private static PipelineDefinition definition() {
return new PipelineDefinition("Sharing Policy", List.of(), OutputSpec.inline());
}
private static Policy watermarkPolicy() {
return new Policy(
POLICY_ID,
"Sharing Policy",
"alice@example.com",
true,
List.of(),
List.of(new PipelineStep("/api/v1/security/add-watermark", Map.of())),
OutputSpec.inline(),
7L);
}
private static ShareEgressDecision unrestricted() {
return ShareEgressDecision.unrestricted(ShareAccessRole.VIEWER);
}
private static ShareEgressDecision transforming() {
return new ShareEgressDecision(
true,
null,
ShareAccessRole.VIEWER,
null,
false,
false,
POLICY_ID,
"Sharing Policy",
List.of(POLICY_ID),
FINGERPRINT);
}
private static ShareEgressDecision viewOnly(List<String> transformPolicyIds) {
return new ShareEgressDecision(
true,
null,
ShareAccessRole.VIEWER,
1,
true,
true,
POLICY_ID,
"Sharing Policy",
transformPolicyIds,
FINGERPRINT);
}
}
@@ -33,6 +33,8 @@ import org.springframework.web.server.ResponseStatusException;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.proprietary.security.database.repository.UserRepository;
import stirling.software.proprietary.security.model.User;
import stirling.software.proprietary.storage.egress.ShareEgressDecision;
import stirling.software.proprietary.storage.egress.ShareEgressPolicyService;
import stirling.software.proprietary.storage.model.FilePurpose;
import stirling.software.proprietary.storage.model.FileShare;
import stirling.software.proprietary.storage.model.FileShareAccess;
@@ -58,6 +60,7 @@ class FileStorageServiceMoreTest {
@Mock private ApplicationProperties applicationProperties;
@Mock private StorageProvider storageProvider;
@Mock private StorageCleanupEntryRepository storageCleanupEntryRepository;
@Mock private ShareEgressPolicyService shareEgressPolicyService;
@Mock private ApplicationProperties.Security securityProperties;
@Mock private ApplicationProperties.System systemProperties;
@@ -77,7 +80,13 @@ class FileStorageServiceMoreTest {
applicationProperties,
storageProvider,
Optional.empty(),
storageCleanupEntryRepository);
storageCleanupEntryRepository,
shareEgressPolicyService);
// No sharing policy configured: every share proceeds exactly as asked.
when(shareEgressPolicyService.evaluateGrant(any(), any(), any(), any(), any()))
.thenAnswer(
invocation -> ShareEgressDecision.unrestricted(invocation.getArgument(4)));
when(applicationProperties.getSecurity()).thenReturn(securityProperties);
when(securityProperties.isEnableLogin()).thenReturn(true);
@@ -3,6 +3,7 @@ package stirling.software.proprietary.storage.service;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.Mockito.atLeastOnce;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.never;
import static org.mockito.Mockito.verify;
@@ -10,11 +11,13 @@ import static org.mockito.Mockito.when;
import java.io.IOException;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.ArgumentCaptor;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import org.mockito.junit.jupiter.MockitoSettings;
@@ -24,10 +27,17 @@ import org.springframework.mock.web.MockMultipartFile;
import org.springframework.web.server.ResponseStatusException;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.proprietary.policy.model.OutputSpec;
import stirling.software.proprietary.policy.model.Policy;
import stirling.software.proprietary.policy.store.InProcessPolicyStore;
import stirling.software.proprietary.policy.store.PolicyStore;
import stirling.software.proprietary.security.database.repository.UserRepository;
import stirling.software.proprietary.security.model.User;
import stirling.software.proprietary.storage.crypto.StorageEncryptionException;
import stirling.software.proprietary.storage.crypto.StorageKeyRevokedException;
import stirling.software.proprietary.storage.egress.ShareChannel;
import stirling.software.proprietary.storage.egress.ShareEgressDecision;
import stirling.software.proprietary.storage.egress.ShareEgressPolicyService;
import stirling.software.proprietary.storage.model.FileShare;
import stirling.software.proprietary.storage.model.ShareAccessRole;
import stirling.software.proprietary.storage.model.StoredFile;
@@ -39,6 +49,8 @@ import stirling.software.proprietary.storage.repository.StorageCleanupEntryRepos
import stirling.software.proprietary.storage.repository.StoredFileRepository;
import stirling.software.proprietary.workflow.model.WorkflowSession;
import tools.jackson.databind.json.JsonMapper;
@ExtendWith(MockitoExtension.class)
@MockitoSettings(strictness = Strictness.LENIENT)
class FileStorageServiceTest {
@@ -50,12 +62,14 @@ class FileStorageServiceTest {
@Mock private ApplicationProperties applicationProperties;
@Mock private StorageProvider storageProvider;
@Mock private StorageCleanupEntryRepository storageCleanupEntryRepository;
@Mock private ShareEgressPolicyService shareEgressPolicyService;
@Mock private ApplicationProperties.Security securityProperties;
@Mock private ApplicationProperties.System systemProperties;
@Mock private ApplicationProperties.Storage storageProperties;
@Mock private ApplicationProperties.Storage.Sharing sharingProperties;
@Mock private ApplicationProperties.Storage.Quotas quotasProperties;
@Mock private ApplicationProperties.Mail mailProperties;
private FileStorageService service;
@@ -70,7 +84,13 @@ class FileStorageServiceTest {
applicationProperties,
storageProvider,
Optional.empty(),
storageCleanupEntryRepository);
storageCleanupEntryRepository,
shareEgressPolicyService);
// No sharing policy configured: every share proceeds exactly as asked.
when(shareEgressPolicyService.evaluateGrant(any(), any(), any(), any(), any()))
.thenAnswer(
invocation -> ShareEgressDecision.unrestricted(invocation.getArgument(4)));
// Default: storage and sharing fully enabled, share links enabled, no expiry
when(applicationProperties.getSecurity()).thenReturn(securityProperties);
@@ -298,6 +318,109 @@ class FileStorageServiceTest {
.isEqualTo(403);
}
@Test
void shareWithUser_registeredUserTypedAsAnEmailAddress_rowIsStillAUserShare() {
User owner = emailUser(1L, "alice@corp.com");
User target = emailUser(2L, "bob@example.com");
StoredFile f = ownedFile(owner);
enableEmailSharing();
when(userRepository.findByUsernameIgnoreCase("bob@example.com"))
.thenReturn(Optional.of(target));
when(fileShareRepository.findByFileAndSharedWithUser(f, target))
.thenReturn(Optional.empty());
when(fileShareRepository.save(any())).thenAnswer(inv -> inv.getArgument(0));
FileShare share =
service.shareWithUser(owner, f, "bob@example.com", ShareAccessRole.VIEWER);
// Usernames are commonly email addresses, so this is the ordinary user-share path: a policy
// narrowed to userShare has to keep biting on it.
assertThat(share.getEgressChannel()).isEqualTo(ShareChannel.USER_SHARE);
verify(shareEgressPolicyService)
.evaluateGrant(
ShareChannel.USER_SHARE,
f,
owner,
"bob@example.com",
ShareAccessRole.VIEWER);
// The link it mails is an email share in its own right.
assertThat(savedShares())
.anyMatch(
saved ->
saved.getShareToken() != null
&& saved.getEgressChannel() == ShareChannel.EMAIL_SHARE);
verify(shareEgressPolicyService)
.evaluateGrant(
ShareChannel.EMAIL_SHARE,
f,
owner,
"bob@example.com",
ShareAccessRole.VIEWER);
}
@Test
void shareWithUser_registeredUserTypedAsAnEmailAddress_deliveryStillEvaluatesAsUserShare() {
User owner = emailUser(1L, "alice@corp.com");
User target = emailUser(2L, "bob@example.com");
StoredFile f = ownedFile(owner);
enableEmailSharing();
when(userRepository.findByUsernameIgnoreCase("bob@example.com"))
.thenReturn(Optional.of(target));
when(fileShareRepository.findByFileAndSharedWithUser(f, target))
.thenReturn(Optional.empty());
when(fileShareRepository.save(any())).thenAnswer(inv -> inv.getArgument(0));
FileShare share =
service.shareWithUser(owner, f, "bob@example.com", ShareAccessRole.VIEWER);
// The real evaluator, on the row this grant wrote: a userShare-only policy must refuse the
// delivery. Stamped EMAIL_SHARE it read as an email share and the policy never bit.
PolicyStore policyStore = new InProcessPolicyStore();
policyStore.save(userShareBlockingPolicy(owner));
ShareEgressPolicyService evaluator =
new ShareEgressPolicyService(policyStore, JsonMapper.builder().build());
assertThat(evaluator.evaluateDelivery(share, target).allowed()).isFalse();
}
/** Every FileShare handed to the repository, in save order. */
private List<FileShare> savedShares() {
ArgumentCaptor<FileShare> captor = ArgumentCaptor.forClass(FileShare.class);
verify(fileShareRepository, atLeastOnce()).save(captor.capture());
return captor.getAllValues();
}
private void enableEmailSharing() {
when(sharingProperties.isEmailEnabled()).thenReturn(true);
when(applicationProperties.getMail()).thenReturn(mailProperties);
when(mailProperties.isEnabled()).thenReturn(true);
}
private User emailUser(long id, String username) {
User u = new User();
u.setId(id);
u.setUsername(username);
return u;
}
/** A Sharing policy that refuses external recipients, narrowed to the userShare channel. */
private Policy userShareBlockingPolicy(User owner) {
return new Policy(
null,
"Sharing Policy",
owner.getUsername(),
true,
List.of(),
List.of(),
new OutputSpec(
"inline",
Map.of(
"categoryId", "sharing",
"sources", List.of("userShare"),
"fieldValues", Map.of("externalRecipients", "block"))),
null);
}
// -------------------------------------------------------------------------
// revokeUserShare
// -------------------------------------------------------------------------
@@ -3034,6 +3034,7 @@ preview = "Preview"
previous = "Previous"
refresh = "Refresh"
remaining = "Remaining"
remove = "Remove"
retry = "Retry"
save = "Save"
stepOf = "Step {{current}} of {{total}}"
@@ -6427,11 +6428,15 @@ enforcingTitle = "Enforcing policy..."
viewAnyway = "View file (policy still enforcing)"
[policyOption]
allow = "Allow"
autoRedactPhi = "Auto-redact PHI"
block = "Block"
blockExport = "Block export"
ccpa = "CCPA"
commenter = "Commenter"
contracts = "Contracts"
documents = "Documents"
editor = "Editor"
fedramp = "FedRAMP"
financialReports = "Financial reports"
flagForReview = "Flag for review"
@@ -6440,6 +6445,7 @@ hipaa = "HIPAA"
hold = "Hold"
hrRecords = "HR records"
indefinite = "Indefinite"
inherit = "Use the system default"
insurance = "Insurance"
invoices = "Invoices"
iso27001 = "ISO 27001"
@@ -6447,6 +6453,7 @@ legalFilings = "Legal filings"
medicalPhi = "Medical / PHI"
never = "Never"
ninetyDays = "90 days"
oneDay = "1 day"
oneYear = "1 year"
p60 = "60%"
p70 = "70%"
@@ -6455,14 +6462,19 @@ p90 = "90%"
p95 = "95%"
pciDss = "PCI DSS"
quarantineDocument = "Quarantine document"
restrict = "Restrict"
restricted = "Restricted"
routeToBucket = "Route to bucket"
s3Bucket = "S3 bucket"
sevenDays = "7 days"
sevenYears = "7 years"
sharePoint = "SharePoint"
soc2 = "SOC 2"
taxDocuments = "Tax documents"
thirtyDays = "30 days"
threeDays = "3 days"
threeYears = "3 years"
viewOnly = "View only"
webhook = "Webhook"
[portal]
@@ -8041,6 +8053,15 @@ label = "Routing"
desc = "Detect PII, redact, strip active content, and watermark documents."
label = "Security"
[portal.policies.categories.sharing]
desc = "Govern how documents are shared: default access, external recipients, and watermarking."
label = "Sharing"
[portal.policies.channels]
emailShare = "Email shares"
shareLink = "Share links"
userShare = "Shares with people"
[portal.policies.config]
scopeAll = "All documents"
@@ -8135,8 +8156,32 @@ summary = "Detects and redacts PII, strips active content (JavaScript), and wate
1 = "Remove JavaScript"
2 = "Watermark"
[portal.policies.config.sharing]
summary = "Governs how stored documents are shared out: who may receive them, what access they get, and what the copy that leaves looks like."
[portal.policies.config.sharing.fields]
defaultAccess = "Default access"
defaultAccessHelper = "The most access any new share may grant. Recipients can be given less, never more."
downloads = "Downloads"
downloadsHelper = "View-only serves a rasterised copy for reading in the browser, never as an attachment download."
externalRecipients = "External recipients"
externalRecipientsHelper = "How to treat recipients outside your organisation. Restrict grants read-only, view-only access on a one-day link."
internalDomains = "Internal domains"
internalDomainsHelper = "Email domains counted as inside your organisation. Leave empty to use the file owner's own domain."
internalDomainsPlaceholder = "example.com"
linkExpiry = "Share link expiry"
linkExpiryHelper = "The longest a share link may live. A policy can only shorten the configured expiry, never extend it."
[portal.policies.config.sharing.rules]
0 = "Default access"
1 = "External recipients"
2 = "Watermark shared copies"
[portal.policies.detail]
atEgress = "At egress, whenever a document is shared"
channels = "Runs on"
enforces = "Enforces"
everyChannel = "Every channel"
onEveryExport = "On every export"
onEveryUpload = "On every upload"
outputAsNewFile = "as a new file"
@@ -8445,6 +8490,11 @@ label = "Add a trusted timestamp"
desc = "Stamps a visible mark (e.g. “Confidential”) across every page."
label = "Apply a watermark"
[portal.policies.wizard.channels]
all = "Every channel unless narrowed."
heading = "Runs on"
narrowed = "Only the channels selected below."
[portal.policies.wizard.classification]
description = "Every uploaded document is classified against the built-in labels and tagged with the types that fit. The label set is shared across your whole team."
labelsHeading = "Classification labels"
@@ -8492,6 +8542,7 @@ setUp = "Set up {{category}} policy"
[portal.policies.wizard.workflow]
description = "Choose what this policy does to every document it processes."
egressDescription = "Choose what happens to the copy that leaves. Recipients get the processed copy; the original is untouched. Leave these off to govern access only."
[portal.policySummary.action]
setUp = "Set up"
@@ -11675,6 +11726,7 @@ alphabet = "Font/Language"
color = "Watermark Colour"
convertToImage = "Flatten PDF pages to images"
opacity = "Opacity (%)"
pickColor = "Pick a colour"
rotation = "Rotation (degrees)"
size = "Size"
@@ -3321,6 +3321,7 @@ preview = "Preview"
previous = "Previous"
refresh = "Refresh"
remaining = "Remaining"
remove = "Remove"
retry = "Retry"
save = "Save"
stepOf = "Step {{current}} of {{total}}"
@@ -6745,11 +6746,15 @@ enforcingTitle = "Enforcing policy..."
viewAnyway = "View file (policy still enforcing)"
[policyOption]
allow = "Allow"
autoRedactPhi = "Auto-redact PHI"
block = "Block"
blockExport = "Block export"
ccpa = "CCPA"
commenter = "Commenter"
contracts = "Contracts"
documents = "Documents"
editor = "Editor"
fedramp = "FedRAMP"
financialReports = "Financial reports"
flagForReview = "Flag for review"
@@ -6758,6 +6763,7 @@ hipaa = "HIPAA"
hold = "Hold"
hrRecords = "HR records"
indefinite = "Indefinite"
inherit = "Use the system default"
insurance = "Insurance"
invoices = "Invoices"
iso27001 = "ISO 27001"
@@ -6765,6 +6771,7 @@ legalFilings = "Legal filings"
medicalPhi = "Medical / PHI"
never = "Never"
ninetyDays = "90 days"
oneDay = "1 day"
oneYear = "1 year"
p60 = "60%"
p70 = "70%"
@@ -6773,14 +6780,19 @@ p90 = "90%"
p95 = "95%"
pciDss = "PCI DSS"
quarantineDocument = "Quarantine document"
restrict = "Restrict"
restricted = "Restricted"
routeToBucket = "Route to bucket"
s3Bucket = "S3 bucket"
sevenDays = "7 days"
sevenYears = "7 years"
sharePoint = "SharePoint"
soc2 = "SOC 2"
taxDocuments = "Tax documents"
thirtyDays = "30 days"
threeDays = "3 days"
threeYears = "3 years"
viewOnly = "View only"
webhook = "Webhook"
[portal]
@@ -8205,6 +8217,9 @@ chooseDestination = "Choose a destination"
chooseOperation = "Choose what this step does"
chooseSource = "Choose a source"
discard = "Discard changes"
editorDestination = "Editor"
editorDestinationDetail = "Replaces the file you ran it on"
editorDestinationHelp = "This pipeline runs on the files in your workspace, and its results replace the file it ran on. There is nowhere else to send them."
inputs = "Input"
inputSource = "Input source"
inputTrigger = "Trigger"
@@ -8216,6 +8231,10 @@ needsSource = "No source chosen"
noToolMatches = "No tools match your search."
pause = "Pause"
rename = "Rename pipeline"
runOn = "Runs on"
runOnExport = "Every export"
runOnTooltip = "Choose when this pipeline runs on your files: when you add them, or when you export them."
runOnUpload = "Every upload"
searchTools = "Search tools"
sendToSystem = "Send to another system"
stepsIncompatible = "These steps can't run on what their prior step produces: {{tools}}."
@@ -8356,6 +8375,8 @@ steps = "Steps"
trigger = "Trigger"
[portal.pipelines.trigger]
editor-export = "Every export"
editor-upload = "Every upload"
folder-watch = "Folder watch"
manual = "Manual"
schedule = "Scheduled"
@@ -8394,6 +8415,15 @@ label = "Routing"
desc = "Detect PII, redact, strip active content, and watermark documents."
label = "Security"
[portal.policies.categories.sharing]
desc = "Govern how documents are shared: default access, external recipients, and watermarking."
label = "Sharing"
[portal.policies.channels]
emailShare = "Email shares"
shareLink = "Share links"
userShare = "Shares with people"
[portal.policies.config]
scopeAll = "All documents"
@@ -8488,8 +8518,32 @@ summary = "Detects and redacts PII, strips active content (JavaScript), and wate
1 = "Remove JavaScript"
2 = "Watermark"
[portal.policies.config.sharing]
summary = "Governs how stored documents are shared out: who may receive them, what access they get, and what the copy that leaves looks like."
[portal.policies.config.sharing.fields]
defaultAccess = "Default access"
defaultAccessHelper = "The most access any new share may grant. Recipients can be given less, never more."
downloads = "Downloads"
downloadsHelper = "View-only serves a rasterised copy for reading in the browser, never as an attachment download."
externalRecipients = "External recipients"
externalRecipientsHelper = "How to treat recipients outside your organization. Restrict grants read-only, view-only access on a one-day link."
internalDomains = "Internal domains"
internalDomainsHelper = "Email domains counted as inside your organization. Leave empty to use the file owner's own domain."
internalDomainsPlaceholder = "example.com"
linkExpiry = "Share link expiry"
linkExpiryHelper = "The longest a share link may live. A policy can only shorten the configured expiry, never extend it."
[portal.policies.config.sharing.rules]
0 = "Default access"
1 = "External recipients"
2 = "Watermark shared copies"
[portal.policies.detail]
atEgress = "At egress, whenever a document is shared"
channels = "Runs on"
enforces = "Enforces"
everyChannel = "Every channel"
onEveryExport = "On every export"
onEveryUpload = "On every upload"
outputAsNewFile = "as a new file"
@@ -8798,6 +8852,11 @@ label = "Add a trusted timestamp"
desc = "Stamps a visible mark (e.g. “Confidential”) across every page."
label = "Apply a watermark"
[portal.policies.wizard.channels]
all = "Every channel unless narrowed."
heading = "Runs on"
narrowed = "Only the channels selected below."
[portal.policies.wizard.classification]
description = "Every uploaded document is classified against the built-in labels and tagged with the types that fit. The label set is shared across your whole team."
labelsHeading = "Classification labels"
@@ -8845,6 +8904,7 @@ setUp = "Set up {{category}} policy"
[portal.policies.wizard.workflow]
description = "Choose what this policy does to every document it processes."
egressDescription = "Choose what happens to the copy that leaves. Recipients get the processed copy; the original is untouched. Leave these off to govern access only."
[portal.policySummary.action]
setUp = "Set up"
@@ -12037,6 +12097,7 @@ alphabet = "Font/Language"
color = "Watermark Color"
convertToImage = "Flatten PDF pages to images"
opacity = "Opacity (%)"
pickColor = "Pick a color"
rotation = "Rotation (degrees)"
size = "Size"
@@ -7,6 +7,7 @@ import ChevronLeftIcon from "@mui/icons-material/ChevronLeft";
import ChevronRightIcon from "@mui/icons-material/ChevronRight";
import { useTranslation } from "react-i18next";
import { getFileSize } from "@app/utils/fileUtils";
import { toolOperationLabel } from "@app/utils/toolOperationLabel";
import { StirlingFileStub } from "@app/types/fileContext";
import { PrivateContent } from "@app/components/shared/PrivateContent";
@@ -115,7 +116,7 @@ const CompactFileDetails: React.FC<CompactFileDetailsProps> = ({
{currentFile?.toolHistory && currentFile.toolHistory.length > 0 && (
<Text size="xs" c="dimmed">
{currentFile.toolHistory
.map((tool) => t(`home.${tool.toolId}.title`, tool.toolId))
.map((tool) => toolOperationLabel(tool, t))
.join(" → ")}
</Text>
)}
@@ -10,7 +10,7 @@ import HistoryIcon from "@mui/icons-material/History";
import MoreVertIcon from "@mui/icons-material/MoreVert";
import { FileId, ToolOperation } from "@app/types/file";
import { ToolId } from "@app/types/toolId";
import { toolOperationLabel } from "@app/utils/toolOperationLabel";
import { StirlingFileStub } from "@app/types/fileContext";
import { formatFileSize, getFileDate } from "@app/utils/fileUtils";
import { downloadFileFromStorage } from "@app/utils/downloadUtils";
@@ -64,10 +64,10 @@ function deltaToolFor(
return curr[priorLen] ?? null;
}
/** Translated tool name via `home.{toolId}.title`. */
function ToolLabel({ toolId }: { toolId: ToolId }) {
/** The operation's own label when it has one, else its translated tool name. */
function ToolLabel({ operation }: { operation: ToolOperation }) {
const { t } = useTranslation();
return <span>{t(`home.${toolId}.title`, toolId)}</span>;
return <span>{toolOperationLabel(operation, t)}</span>;
}
export interface VersionTimelineProps {
@@ -242,7 +242,7 @@ export function VersionTimeline({
style={{ color: "var(--c-text)" }}
>
{delta ? (
<ToolLabel toolId={delta.toolId} />
<ToolLabel operation={delta} />
) : (
t("filesPage.versionOrigin", "Original upload")
)}
@@ -10,6 +10,7 @@ import LabelOutlinedIcon from "@mui/icons-material/LabelOutlined";
import CheckCircleOutlinedIcon from "@mui/icons-material/CheckCircleOutlined";
import AltRouteOutlinedIcon from "@mui/icons-material/AltRouteOutlined";
import ScheduleOutlinedIcon from "@mui/icons-material/ScheduleOutlined";
import IosShareOutlinedIcon from "@mui/icons-material/IosShareOutlined";
type MuiIcon = React.ComponentType<{ sx?: SxProps<Theme>; className?: string }>;
@@ -18,6 +19,7 @@ const POLICY_CATEGORY_ICONS: Record<string, MuiIcon> = {
ingestion: LayersOutlinedIcon,
security: ShieldOutlinedIcon,
classification: LabelOutlinedIcon,
sharing: IosShareOutlinedIcon,
compliance: CheckCircleOutlinedIcon,
routing: AltRouteOutlinedIcon,
retention: ScheduleOutlinedIcon,
@@ -6,8 +6,8 @@
import React from "react";
import { Text, Tooltip, Badge, Group } from "@mantine/core";
import { ToolOperation } from "@app/types/file";
import { toolOperationLabel } from "@app/utils/toolOperationLabel";
import { useTranslation } from "react-i18next";
import { ToolId } from "@app/types/toolId";
interface ToolChainProps {
toolChain: ToolOperation[];
@@ -29,11 +29,7 @@ const ToolChain: React.FC<ToolChainProps> = ({
const { t } = useTranslation();
if (!toolChain || toolChain.length === 0) return null;
const toolIds = toolChain.map((tool) => tool.toolId);
const getToolName = (toolId: ToolId) => {
return t(`home.${toolId}.title`, toolId);
};
const getToolName = (tool: ToolOperation) => toolOperationLabel(tool, t);
// Create full tool chain for tooltip
const fullChainDisplay =
@@ -42,7 +38,7 @@ const ToolChain: React.FC<ToolChainProps> = ({
{toolChain.map((tool, index) => (
<React.Fragment key={`${tool.toolId}-${index}`}>
<Badge size="sm" variant="light" color="blue">
{getToolName(tool.toolId)}
{getToolName(tool)}
</Badge>
{index < toolChain.length - 1 && (
<Text size="sm" c="dimmed">
@@ -53,18 +49,21 @@ const ToolChain: React.FC<ToolChainProps> = ({
))}
</Group>
) : (
<Text size="sm">{toolIds.map(getToolName).join(" → ")}</Text>
<Text size="sm">{toolChain.map(getToolName).join(" → ")}</Text>
);
// Create truncated display based on available space
const getTruncatedDisplay = () => {
if (toolIds.length <= 2) {
if (toolChain.length <= 2) {
// Show all tools if 2 or fewer
return { text: toolIds.map(getToolName).join(" → "), isTruncated: false };
return {
text: toolChain.map(getToolName).join(" → "),
isTruncated: false,
};
} else {
// Show first tool ... last tool for longer chains
return {
text: `${getToolName(toolIds[0])} → +${toolIds.length - 2}${getToolName(toolIds[toolIds.length - 1])}`,
text: `${getToolName(toolChain[0])} → +${toolChain.length - 2}${getToolName(toolChain[toolChain.length - 1])}`,
isTruncated: true,
};
}
@@ -75,10 +74,10 @@ const ToolChain: React.FC<ToolChainProps> = ({
// Compact style for very small spaces
if (displayStyle === "compact") {
const compactText =
toolIds.length === 1
? getToolName(toolIds[0])
: `${toolIds.length} tools`;
const isCompactTruncated = toolIds.length > 1;
toolChain.length === 1
? getToolName(toolChain[0])
: `${toolChain.length} tools`;
const isCompactTruncated = toolChain.length > 1;
const compactElement = (
<Text
@@ -116,7 +115,7 @@ const ToolChain: React.FC<ToolChainProps> = ({
{toolChain.slice(0, 3).map((tool, index) => (
<React.Fragment key={`${tool.toolId}-${index}`}>
<Badge size={size} variant="light" color="blue">
{getToolName(tool.toolId)}
{getToolName(tool)}
</Badge>
{index < Math.min(toolChain.length - 1, 2) && (
<Text size="xs" c="dimmed">
@@ -131,7 +130,7 @@ const ToolChain: React.FC<ToolChainProps> = ({
...
</Text>
<Badge size={size} variant="light" color="blue">
{getToolName(toolChain[toolChain.length - 1].toolId)}
{getToolName(toolChain[toolChain.length - 1])}
</Badge>
</>
)}
@@ -140,7 +139,7 @@ const ToolChain: React.FC<ToolChainProps> = ({
);
return isBadgesTruncated ? (
<Tooltip label={`${toolIds.map(getToolName).join(" → ")}`} withinPortal>
<Tooltip label={`${toolChain.map(getToolName).join(" → ")}`} withinPortal>
{badgesElement}
</Tooltip>
) : (
@@ -35,6 +35,10 @@ const WatermarkTextStyle = ({
onChange={(value) => onParameterChange("customColor", value)}
disabled={disabled}
format="hex"
// Mantine's eyedropper is an icon-only button with no accessible name.
eyeDropperButtonProps={{
"aria-label": t("watermark.settings.pickColor", "Pick a colour"),
}}
popoverProps={{
withinPortal: true,
zIndex: Z_INDEX_AUTOMATE_DROPDOWN,
@@ -211,7 +211,7 @@ describe("fileContextReducer — silent CONSUME_FILES (background enforcement)",
it("carries classificationConfidence forward with the labels", () => {
// The confidence is part of the verdict: without it the escalation decision
// (shouldDispatchToAi) dies at the version boundary and a chained
// (localVerdictNeedsEscalation) dies at the version boundary and a chained
// classification never runs.
const start = stateWith([
stub("a", {
@@ -390,8 +390,8 @@ export function fileContextReducer(
// tool that versions/derives a classified file keeps it in its label
// groups instead of dropping to "Other" and waiting on a PDF re-read.
// Inherited from the first input that has labels, together with that
// verdict's confidence - the escalation decision (shouldDispatchToAi) is
// about the document, not about which step produced the current bytes, so
// verdict's confidence - the escalation decision (localVerdictNeedsEscalation)
// is about the document, not about which step produced the current bytes, so
// it must survive the version boundary. An output that already carries its
// own verdict (e.g. a fresh classify result) keeps it.
const verdictDonor = inputFileIds
@@ -14,6 +14,8 @@ export async function createStirlingFilesAndStubs(
files: File[],
parentStub: StirlingFileStub,
toolId: ToolId,
/** Shown instead of the tool's name in version history (a policy passes its pipeline name). */
label?: string,
): Promise<{ stirlingFiles: StirlingFile[]; stubs: StirlingFileStub[] }> {
const stirlingFiles: StirlingFile[] = [];
const stubs: StirlingFileStub[] = [];
@@ -22,7 +24,7 @@ export async function createStirlingFilesAndStubs(
const processedFileMetadata = await generateProcessedFileMetadata(file);
const childStub = createChildStub(
parentStub,
{ toolId, timestamp: Date.now() },
{ toolId, timestamp: Date.now(), ...(label ? { label } : {}) },
file,
processedFileMetadata?.thumbnailUrl,
processedFileMetadata,
@@ -12,14 +12,20 @@ const FIXTURES = path.join(
"../test-fixtures/classification/unlabelled",
);
/** The stored policy DefaultClassificationPolicySeeder writes for a new team. */
/**
* What GET /api/v1/policies returns for the row an older
* DefaultClassificationPolicySeeder wrote - i.e. one stored before editor
* participation had its own field. `JpaPolicyStore.liftEditorConfig` derives the
* `editor` block from the legacy `output.options` on read; the lift is additive,
* so a real response carries both. Migration of the stored shape itself is
* covered by JpaPolicyStoreTest, which exercises the Java the stub stands in for.
*/
const SEEDED_POLICY = {
id: "seeded-classification",
name: "Classification Policy",
owner: "system",
enabled: true,
trigger: null,
sourceIds: [],
inputs: [],
steps: [{ operation: "/api/v1/ai/tools/classify-and-label", parameters: {} }],
output: {
type: "inline",
@@ -32,7 +38,9 @@ const SEEDED_POLICY = {
reviewerEmail: "",
},
},
outputIds: [],
teamId: 1,
editor: { allowed: true, runOn: "upload" },
};
test("a 10-file upload wave classifies every file into its group", async ({
@@ -0,0 +1,85 @@
import path from "path";
import { test, expect } from "@app/tests/helpers/stub-test-base";
import { uploadFiles } from "@app/tests/helpers/ui-helpers";
// A pipeline reaches the editor auto-run through its own editor flag; a swept one must not.
test.use({ autoGoto: false });
const SAMPLE = path.join(
import.meta.dirname,
"../test-fixtures/classification/unlabelled/invoice_acme.pdf",
);
/** A builder-made pipeline: no categoryId, one harmless step. */
function builderPipeline(editor: { allowed: boolean; runOn: string }) {
return {
id: "builder-pipeline-1",
name: "Flatten everything",
owner: "system",
enabled: true,
trigger: null,
sourceIds: [],
steps: [{ operation: "/api/v1/misc/flatten", parameters: {} }],
output: { type: "inline", options: { mode: "new_version" } },
editor,
teamId: 1,
};
}
/** Install the policy list + capture every stored-policy run dispatch. */
async function armed(page: import("@playwright/test").Page, policy: unknown) {
const dispatched: string[] = [];
await page.route("**/api/v1/policies", (route) =>
route.fulfill({ json: [policy] }),
);
await page.route("**/api/v1/policies/*/run", (route) => {
dispatched.push(new URL(route.request().url()).pathname);
return route.fulfill({ json: { jobId: "job-1" } });
});
return dispatched;
}
test("an editor pipeline set to run on upload dispatches when a file is added", async ({
page,
}) => {
const dispatched = await armed(
page,
builderPipeline({ allowed: true, runOn: "upload" }),
);
await page.goto("/editor", { waitUntil: "domcontentloaded" });
await uploadFiles(page, SAMPLE);
await expect
.poll(() => dispatched, { timeout: 15_000 })
.toContain("/api/v1/policies/builder-pipeline-1/run");
});
test("a swept pipeline never runs on editor upload", async ({ page }) => {
const dispatched = await armed(
page,
builderPipeline({ allowed: false, runOn: "upload" }),
);
await page.goto("/editor", { waitUntil: "domcontentloaded" });
await uploadFiles(page, SAMPLE);
await page.waitForTimeout(5_000);
expect(dispatched).toEqual([]);
});
test("an editor pipeline set to run on export does not fire on upload", async ({
page,
}) => {
const dispatched = await armed(
page,
builderPipeline({ allowed: true, runOn: "export" }),
);
await page.goto("/editor", { waitUntil: "domcontentloaded" });
await uploadFiles(page, SAMPLE);
await page.waitForTimeout(5_000);
expect(dispatched).toEqual([]);
});
+3
View File
@@ -16,6 +16,9 @@ export type FileId = string & { readonly [tag]: "FileId" };
export interface ToolOperation {
toolId: ToolId;
timestamp: number;
/** Overrides the tool's own name in history. Set by a policy run to its pipeline's name, since
* every policy records the same "automate" toolId. */
label?: string;
}
/**
+4
View File
@@ -33,6 +33,8 @@ export interface ChipProps extends Omit<
/** Shows a spinner and dims the chip. */
loading?: boolean;
onRemove?: () => void;
/** Accessible name for the remove button; the icon alone gives it none. */
removeLabel?: string;
onClick?: () => void;
/** Leading status dot. Use for status-style chips. */
showDot?: boolean;
@@ -52,6 +54,7 @@ export function Chip({
trailingIcon,
loading = false,
onRemove,
removeLabel,
onClick,
showDot,
dashed,
@@ -104,6 +107,7 @@ export function Chip({
size={size}
withRemoveButton={removable}
onRemove={onRemove}
removeButtonProps={{ "aria-label": removeLabel ?? "Remove" }}
{...rootProps}
>
{showDot && <span className="sui-chip__dot" aria-hidden />}
@@ -0,0 +1,29 @@
import { describe, it, expect } from "vitest";
import type { TFunction } from "i18next";
import { toolOperationLabel } from "@app/utils/toolOperationLabel";
import type { ToolOperation } from "@app/types/file";
// Stands in for i18next: echoes the key so the assertions show which lookup ran.
const t = ((key: string, fallback?: string) =>
key === "home.automate.title" ? "Automate" : (fallback ?? key)) as TFunction;
const op = (over: Partial<ToolOperation>): ToolOperation =>
({ toolId: "automate", timestamp: 0, ...over }) as ToolOperation;
describe("toolOperationLabel", () => {
it("prefers the operation's own label", () => {
expect(toolOperationLabel(op({ label: "add-page-numbers" }), t)).toBe(
"add-page-numbers",
);
});
// Every policy records the same "automate" toolId, so without a label each automated version
// reads identically no matter which pipeline produced it.
it("falls back to the tool's name when unlabelled", () => {
expect(toolOperationLabel(op({}), t)).toBe("Automate");
});
it("keeps the fallback for an empty label rather than rendering a blank", () => {
expect(toolOperationLabel(op({ label: "" }), t)).toBe("Automate");
});
});
@@ -0,0 +1,17 @@
import type { TFunction } from "i18next";
import type { ToolOperation } from "@app/types/file";
/**
* What produced a version, for the history surfaces. A policy run carries its own label (the
* pipeline's name) because every policy records the same "automate" toolId, which would otherwise
* render every automated version identically.
*/
export function toolOperationLabel(
operation: ToolOperation,
t: TFunction,
): string {
// Truthiness, not nullish: a blank label would otherwise render as an empty history entry.
return (
operation.label || t(`home.${operation.toolId}.title`, operation.toolId)
);
}
@@ -70,6 +70,8 @@ export interface Policy {
* output} is used.
*/
outputIds: string[];
/** Whether the editor runs this policy per file, and on which moment. */
editor?: { allowed: boolean; runOn: "upload" | "export" };
teamId?: number | null;
}
@@ -0,0 +1,126 @@
import { describe, it, expect } from "vitest";
import {
assemblePolicies,
POLICY_CATEGORIES,
POLICY_CONFIG,
SHARE_CHANNELS,
buildWireFromSetup,
type CatalogueEntry,
} from "@portal/api/policies";
const t = ((key: string) => key) as unknown as Parameters<
typeof buildWireFromSetup
>[2];
describe("policy catalogue", () => {
it("gives every category a config", () => {
// decoratePolicy silently drops a policy whose category has no config, so a
// category added without one would just never appear.
for (const category of POLICY_CATEGORIES) {
expect(POLICY_CONFIG[category.id], category.id).toBeDefined();
}
});
it("carries a sharing category that runs at egress", () => {
const sharing = POLICY_CATEGORIES.find((c) => c.id === "sharing");
expect(sharing).toBeDefined();
expect(sharing?.runsAtEgress).toBe(true);
expect(sharing?.comingSoon).toBeUndefined();
});
it("defaults the sharing policy to the restrictive settings", () => {
const fields = Object.fromEntries(
POLICY_CONFIG.sharing.fields.map((f) => [f.key, f.value]),
);
expect(fields.defaultAccess).toBe("restricted");
expect(fields.externalRecipients).toBe("restrict");
expect(fields.downloads).toBe("allow");
expect(fields.internalDomains).toEqual([]);
});
it("watermarks the outgoing copy by default and offers redaction alongside", () => {
const tools = POLICY_CONFIG.sharing.defaultOperations.map((s) => s.toolId);
expect(tools[0]).toBe("watermark");
expect(tools).toContain("redact");
expect(tools).toContain("sanitize");
// Marking the copy is the promised default; scrubbing it is opt-in.
expect(POLICY_CONFIG.sharing.defaultOn).toEqual(["watermark"]);
});
it("ships watermark text so the default policy works unconfigured", () => {
// The watermark tool rejects an empty text, so a preset without one would
// fail every run of a freshly created policy.
const watermark = POLICY_CONFIG.sharing.defaultOperations.find(
(s) => s.toolId === "watermark",
);
expect(watermark?.params).toMatchObject({
watermarkType: "text",
watermarkText: expect.stringMatching(/\S/),
});
});
});
describe("share channels", () => {
it("uses the ids the backend ShareChannel enum matches on", () => {
// These are persisted in the policy's `sources` list and parsed server-side
// by ShareChannel.fromId, so they are part of the stored format.
expect(SHARE_CHANNELS.map((c) => c.id)).toEqual([
"userShare",
"shareLink",
"emailShare",
]);
});
});
describe("sharing policy round-trip", () => {
const entry: CatalogueEntry = {
category: POLICY_CATEGORIES.find((c) => c.id === "sharing")!,
config: POLICY_CONFIG.sharing,
policy: null,
};
const wire = buildWireFromSetup(
entry,
{
fieldValues: { defaultAccess: "restricted", externalRecipients: "block" },
sources: ["shareLink"],
// An egress policy is enforced server-side on delivery, never on the editor.
runsOnEditor: false,
scopeTypes: ["contract", "nda"],
reviewerEmail: "",
outputMode: "new_version",
outputName: "",
outputNamePosition: "suffix",
runOn: "upload",
maxRetries: 0,
retryDelayMinutes: 0,
steps: [{ operation: "/api/v1/security/add-watermark", parameters: {} }],
},
t,
);
it("carries no trigger - the backend finds egress policies by category", () => {
expect(wire.trigger).toBeNull();
});
it("persists the channels the backend narrows on", () => {
expect(wire.output.options.sources).toEqual(["shareLink"]);
expect(wire.output.options.categoryId).toBe("sharing");
});
it("round-trips scopeTypes untouched even though nothing sets it yet", () => {
// No document-type UI ships today, but the field stays in the wire format so
// an existing value survives an edit rather than being silently dropped.
expect(wire.output.options.scopeTypes).toEqual(["contract", "nda"]);
});
it("survives the catalogue assembly it will be read back through", () => {
const assembled = assemblePolicies([wire], []);
const sharing = assembled.catalogue.find(
(e) => e.category.id === "sharing",
);
expect(sharing?.policy).not.toBeNull();
expect(sharing?.policy?.state.sources).toEqual(["shareLink"]);
expect(sharing?.policy?.state.scopeTypes).toEqual(["contract", "nda"]);
});
});
+107 -2
View File
@@ -14,7 +14,11 @@ import { apiClient } from "@portal/api/http";
import { fromWirePolicy, toWirePolicy } from "@app/policies/codec";
import { resolveRunOn } from "@app/policies/runOn";
import { runsToActivity, runsToStats } from "@app/policies/runs";
import { policyStep, type PolicyToolStep } from "@app/policies/operations";
import {
policyStep,
type PolicyToolId,
type PolicyToolStep,
} from "@app/policies/operations";
import type { ToolEndpoint } from "@app/types/toolApiTypes";
import type {
PolicyDecodedState,
@@ -44,7 +48,7 @@ export type PolicyStatus = "active" | "paused";
export type PolicyRowStatus = "active" | "paused" | "setup";
export type PolicyFieldType = "toggle" | "select" | "chips" | "text";
export type PolicyFieldType = "toggle" | "select" | "chips" | "text" | "tags";
export interface PolicyField {
label: string;
@@ -52,6 +56,10 @@ export interface PolicyField {
type: PolicyFieldType;
value: boolean | string | string[];
options?: string[];
/** Shown under the control. An i18n key, like `label`. */
helper?: string;
/** Placeholder for the free-text entry of a `tags` field. An i18n key. */
placeholder?: string;
}
export interface PolicyCategory {
@@ -62,6 +70,9 @@ export interface PolicyCategory {
providesClassification?: boolean;
comingSoon?: boolean;
requiresAiEngine?: boolean;
/** Fires when a document leaves (a share), not on upload/export: enforced server-side, share
* channels instead of sources, and valid with no tool chain at all. */
runsAtEgress?: boolean;
}
export interface PolicyConfigDef {
@@ -70,12 +81,17 @@ export interface PolicyConfigDef {
scopeLabel: string;
fields: PolicyField[];
defaultOperations: PolicyToolStep[];
/** Which of `defaultOperations` start switched on. Absent falls back to the wizard's global
* "off until configured" set, which suits presets that can't supply parameters. */
defaultOn?: PolicyToolId[];
}
export interface PolicyState {
configured: boolean;
status: PolicyStatus;
sources: string[];
/** Whether the editor runs this policy per file; stored, not derived from `sources`. */
runsOnEditor?: boolean;
scopeTypes: string[];
reviewerEmail: string;
fieldValues: Record<string, boolean | string | string[]>;
@@ -92,6 +108,7 @@ export interface PolicyState {
export interface PolicySetupResult {
fieldValues: Record<string, boolean | string | string[]>;
sources: string[];
runsOnEditor: boolean;
scopeTypes: string[];
reviewerEmail: string;
outputMode: "new_file" | "new_version";
@@ -196,6 +213,13 @@ export const POLICY_CATEGORIES: PolicyCategory[] = [
desc: "portal.policies.categories.classification.desc",
providesClassification: true,
},
{
id: "sharing",
label: "portal.policies.categories.sharing.label",
tone: "green",
desc: "portal.policies.categories.sharing.desc",
runsAtEgress: true,
},
{
id: "compliance",
label: "portal.policies.categories.compliance.label",
@@ -284,6 +308,75 @@ export const POLICY_CONFIG: Record<string, PolicyConfigDef> = {
defaultOperations: [policyStep("classify")],
fields: [],
},
sharing: {
summary: "portal.policies.config.sharing.summary",
rules: [
"portal.policies.config.sharing.rules.0",
"portal.policies.config.sharing.rules.1",
"portal.policies.config.sharing.rules.2",
],
scopeLabel: "portal.policies.config.scopeAll",
// Watermark is on by default with real text so it works unconfigured, baked in via image so
// it can't be lifted off. Redact/sanitise stay opt-in: a bigger change than a mark.
defaultOperations: [
policyStep("watermark", {
watermarkType: "text",
watermarkText: "Confidential",
convertPDFToImage: true,
}),
policyStep("redact", {
useRegex: true,
convertPDFToImage: true,
wordsToRedact: DEFAULT_PII_PATTERNS,
}),
policyStep("sanitize", { removeEmbeddedFiles: false }),
],
defaultOn: ["watermark"],
fields: [
{
label: "portal.policies.config.sharing.fields.defaultAccess",
key: "defaultAccess",
type: "select",
value: "restricted",
options: ["restricted", "commenter", "editor", "inherit"],
helper: "portal.policies.config.sharing.fields.defaultAccessHelper",
},
{
label: "portal.policies.config.sharing.fields.externalRecipients",
key: "externalRecipients",
type: "select",
value: "restrict",
options: ["allow", "restrict", "block"],
helper:
"portal.policies.config.sharing.fields.externalRecipientsHelper",
},
{
label: "portal.policies.config.sharing.fields.internalDomains",
key: "internalDomains",
type: "tags",
value: [],
helper: "portal.policies.config.sharing.fields.internalDomainsHelper",
placeholder:
"portal.policies.config.sharing.fields.internalDomainsPlaceholder",
},
{
label: "portal.policies.config.sharing.fields.linkExpiry",
key: "linkExpiry",
type: "select",
value: "sevenDays",
options: ["oneDay", "threeDays", "sevenDays", "thirtyDays", "inherit"],
helper: "portal.policies.config.sharing.fields.linkExpiryHelper",
},
{
label: "portal.policies.config.sharing.fields.downloads",
key: "downloads",
type: "select",
value: "allow",
options: ["allow", "viewOnly"],
helper: "portal.policies.config.sharing.fields.downloadsHelper",
},
],
},
compliance: {
summary: "portal.policies.config.compliance.summary",
rules: [
@@ -416,6 +509,14 @@ export const POLICY_DOC_TYPES: string[] = [
"financialReports",
];
/** The ways a document leaves today. Persisted in the policy's `sources` and matched server-side
* by `ShareChannel`, so these ids must stay in step with that enum. */
export const SHARE_CHANNELS: { id: string; label: string }[] = [
{ id: "userShare", label: "portal.policies.channels.userShare" },
{ id: "shareLink", label: "portal.policies.channels.shareLink" },
{ id: "emailShare", label: "portal.policies.channels.emailShare" },
];
// ── Client-side catalogue assembly ───────────────────────────────────────────
function decoratePolicy(
@@ -433,6 +534,7 @@ function decoratePolicy(
configured: true,
status,
sources: decoded.sources,
runsOnEditor: decoded.runsOnEditor,
scopeTypes: decoded.scopeTypes,
reviewerEmail: decoded.reviewerEmail,
fieldValues: decoded.fieldValues,
@@ -595,6 +697,7 @@ export function buildWireFromSetup(
enabled,
categoryId: entry.category.id,
sources: result.sources,
runsOnEditor: result.runsOnEditor,
scopeTypes: result.scopeTypes,
reviewerEmail: result.reviewerEmail,
fieldValues: result.fieldValues,
@@ -625,6 +728,8 @@ export function buildWireFromState(
enabled,
categoryId: entry.category.id,
sources: s.sources,
// Carry the stored value through: pause/resume must not re-derive it.
runsOnEditor: s.runsOnEditor === true,
scopeTypes: s.scopeTypes,
reviewerEmail: s.reviewerEmail,
fieldValues: s.fieldValues,
@@ -0,0 +1,140 @@
// Swept sources are scheduled or triggered server-side; the editor runs client-side as each file
// passes through, so the two get different controls.
import { useTranslation } from "react-i18next";
import { Tooltip } from "@mantine/core";
import InfoOutlinedIcon from "@mui/icons-material/InfoOutlined";
import { FormField, Input, Select } from "@app/ui";
export type ScheduleUnit = "MINUTES" | "HOURS" | "DAYS";
export type EditorRunOn = "upload" | "export";
const SCHEDULE_UNITS: ScheduleUnit[] = ["MINUTES", "HOURS", "DAYS"];
/** Empty trigger type = manual-only (no automatic trigger). */
export const MANUAL = "";
/** Sentinel for manual: Mantine's Select reads "" as no selection. Maps to {@link MANUAL}. */
export const MANUAL_OPTION = "manual";
/** One input row in the builder: a source paired with its own trigger config. */
export interface WorkingInput {
sourceId: string;
triggerType: string;
scheduleCount: string;
scheduleUnit: ScheduleUnit;
}
export interface PipelineInputTriggerProps {
input: WorkingInput;
onInputChange: (patch: Partial<WorkingInput>) => void;
/** Trigger types offered for this row's source (manual first). */
triggerOptions: { value: string; label: string }[];
/** The chosen source is the editor, so the pipeline runs in the browser. */
isEditorInput: boolean;
runOn: EditorRunOn;
onRunOnChange: (runOn: EditorRunOn) => void;
}
export function PipelineInputTrigger({
input,
onInputChange,
triggerOptions,
isEditorInput,
runOn,
onRunOnChange,
}: PipelineInputTriggerProps) {
const { t } = useTranslation();
if (isEditorInput) {
const label = t("portal.pipelines.builder.runOn", "Runs on");
return (
<FormField
label={
<Tooltip
label={t(
"portal.pipelines.builder.runOnTooltip",
"Choose when this pipeline runs on your files: when you add them, or when you export them.",
)}
position="right"
withinPortal
multiline
w={260}
>
<span className="portal-builder__label-hint">
{label}
<InfoOutlinedIcon style={{ fontSize: "0.875rem" }} />
</span>
</Tooltip>
}
>
<Select
inputSize="sm"
aria-label={label}
value={runOn}
onChange={(value) =>
onRunOnChange(value === "export" ? "export" : "upload")
}
options={[
{
value: "upload",
label: t("portal.pipelines.builder.runOnUpload", "Every upload"),
},
{
value: "export",
label: t("portal.pipelines.builder.runOnExport", "Every export"),
},
]}
/>
</FormField>
);
}
return (
<>
<FormField label={t("portal.pipelines.builder.inputTrigger")}>
<Select
inputSize="sm"
aria-label={t("portal.pipelines.builder.inputTrigger")}
value={
input.triggerType === MANUAL ? MANUAL_OPTION : input.triggerType
}
disabled={input.sourceId === ""}
onChange={(value) =>
onInputChange({
triggerType: value && value !== MANUAL_OPTION ? value : MANUAL,
})
}
options={triggerOptions}
/>
</FormField>
{input.triggerType === "schedule" && (
<div className="portal-builder__schedule">
<span className="portal-builder__muted">
{t("portal.pipelines.composer.scheduleEvery")}
</span>
<Input
inputSize="sm"
type="number"
min={1}
value={input.scheduleCount}
invalid={Number(input.scheduleCount) <= 0}
onChange={(e) => onInputChange({ scheduleCount: e.target.value })}
className="portal-builder__schedule-count"
/>
<Select
inputSize="sm"
value={input.scheduleUnit}
onChange={(value) =>
value && onInputChange({ scheduleUnit: value as ScheduleUnit })
}
options={SCHEDULE_UNITS.map((unit) => ({
value: unit,
label: t(`portal.pipelines.composer.unit.${unit.toLowerCase()}`),
}))}
/>
</div>
)}
</>
);
}
@@ -51,6 +51,8 @@ export interface GraphNodeContent {
warning?: string;
/** Why the input will not be much use. */
inputWarning?: ChainWarning;
/** An end the pipeline decides for itself, so it carries no remove control. */
fixed?: boolean;
}
export interface GraphStepContent extends GraphNodeContent {
@@ -294,7 +296,9 @@ export function PipelineGraph({
warning={content.warning}
selected={selected === kind}
onSelect={() => onSelect(kind)}
onRemove={() => onRemoveEnd(kind)}
onRemove={
content.fixed ? undefined : () => onRemoveEnd(kind)
}
/>
)}
</div>
@@ -88,7 +88,7 @@ export function PolicyCatalogueTable({
},
}),
],
[t, onOpen, isLocked, lockedLabel],
[t, isLocked, lockedLabel],
);
return (
@@ -1,5 +1,8 @@
import type { Meta, StoryObj } from "@storybook/react-vite";
import { decorateForStory } from "@portal/components/policies/storyFixtures";
import {
decorateForStory,
decorateSharingForStory,
} from "@portal/components/policies/storyFixtures";
import { PolicyDetailPanel } from "@portal/components/policies/PolicyDetailPanel";
const meta: Meta<typeof PolicyDetailPanel> = {
@@ -33,6 +36,21 @@ export const Paused: Story = {
},
};
/** An egress policy: the trigger strip reads "at egress" and "Runs on" lists share channels. */
export const SharingEveryChannel: Story = {
args: { policy: decorateSharingForStory() },
};
/** The same policy narrowed to share links and to two document types. */
export const SharingNarrowed: Story = {
args: {
policy: decorateSharingForStory({
sources: ["shareLink", "emailShare"],
scopeTypes: ["contract", "nda"],
}),
},
};
/** A custom (deletable) policy with no runs yet — empty activity feed. */
export const CustomNoActivity: Story = {
args: {
@@ -10,6 +10,7 @@ import {
} from "@app/ui";
import {
humanizeEndpoint,
SHARE_CHANNELS,
type DecoratedPolicy,
type PolicyActivityItem,
} from "@portal/api/policies";
@@ -115,13 +116,18 @@ export function PolicyDetailPanel({
if (!policy) return null;
const { category, config, state, steps, stats, activity } = policy;
const isPaused = state.status === "paused";
const isEgress = category.runsAtEgress === true;
const canDelete = state.isDefault !== true;
// Processed history only exists for watched sources; editor uploads are never ledgered.
// Editor participation is its own flag (runsOnEditor), not a source. A legacy policy still carries
// "editor" in its stored sources until re-saved, so drop it here to count only real watched sources.
const realSources = state.sources.filter((s) => s !== "editor");
// Processed history only exists for watched sources; editor uploads are never ledgered, and an
// egress policy's "sources" are channels, not files it sweeps.
const canClearHistory =
onClearHistory !== undefined && state.sources.some((s) => s !== "editor");
onClearHistory !== undefined && !isEgress && realSources.length > 0;
const enforceItems = steps.length > 0 ? steps.map((s) => s.operation) : null;
const hasEditorSource = state.sources.includes("editor");
const hasEditorSource = state.runsOnEditor === true;
const trigger =
state.runOn === "export"
? t("portal.policies.detail.onEveryExport")
@@ -133,7 +139,8 @@ export function PolicyDetailPanel({
function sourceLabel(id: string) {
if (id === "editor") return t("portal.sources.types.editor.label");
return id;
const channel = SHARE_CHANNELS.find((c) => c.id === id);
return channel ? t(channel.label) : id;
}
return (
@@ -157,7 +164,9 @@ export function PolicyDetailPanel({
{t("portal.policies.detail.actions.delete")}
</Button>
)}
{onRun && (
{/* An egress policy has nothing to run on demand: it fires when a
document is shared, not against a set of files. */}
{onRun && !isEgress && (
<Button
variant="secondary"
size="sm"
@@ -201,7 +210,17 @@ export function PolicyDetailPanel({
? t("portal.policies.status.paused")
: t("portal.policies.status.active")}
</StatusBadge>
{hasEditorSource && (
{isEgress && (
<>
<span className="portal-policies__detail-sep" aria-hidden>
·
</span>
<span className="portal-policies__detail-meta">
{t("portal.policies.detail.atEgress")}
</span>
</>
)}
{!isEgress && hasEditorSource && (
<>
<span className="portal-policies__detail-sep" aria-hidden>
·
@@ -242,14 +261,18 @@ export function PolicyDetailPanel({
</span>
</div>
{/* Sources */}
{state.sources.length > 0 && (
{/* Runs on — input sources, or share channels for an egress policy. */}
{(realSources.length > 0 || isEgress) && (
<div className="portal-policies__detail-inline">
<span className="portal-policies__detail-inline-label">
{t("portal.policies.detail.sources")}
{isEgress
? t("portal.policies.detail.channels")
: t("portal.policies.detail.sources")}
</span>
<span className="portal-policies__detail-inline-value">
{state.sources.map(sourceLabel).join(" · ")}
{realSources.length > 0
? realSources.map(sourceLabel).join(" · ")
: t("portal.policies.detail.everyChannel")}
</span>
</div>
)}
@@ -1,3 +1,4 @@
import { useState, type KeyboardEvent } from "react";
import { useTranslation } from "react-i18next";
import { Chip, FormField, Input, Select, ToggleSwitch } from "@app/ui";
import type { PolicyField } from "@portal/api/policies";
@@ -10,17 +11,16 @@ interface PolicyFieldRowProps {
onChange: (value: boolean | string | string[]) => void;
}
/**
* Renders one policy setting from the catalogue's `PolicyField`, dispatching on
* `type`: toggle → ToggleSwitch, select → Select, chips → multi-select Chips,
* text → Input. Controlled — the setup flow owns the value.
*/
/** Renders one `PolicyField`, dispatching on `type`. Controlled — the setup flow owns the
* value. */
export function PolicyFieldRow({
field,
value,
onChange,
}: PolicyFieldRowProps) {
const { t } = useTranslation();
const helper = field.helper ? t(field.helper) : undefined;
if (field.type === "toggle") {
return (
<div className="portal-policies__toggle-row">
@@ -28,11 +28,24 @@ export function PolicyFieldRow({
checked={Boolean(value)}
onChange={onChange}
label={t(field.label)}
description={helper}
/>
</div>
);
}
if (field.type === "tags") {
return (
<FormField label={t(field.label)} helperText={helper}>
<PolicyTagsField
value={Array.isArray(value) ? value : []}
placeholder={field.placeholder ? t(field.placeholder) : undefined}
onChange={onChange}
/>
</FormField>
);
}
if (field.type === "chips") {
const selected = Array.isArray(value) ? value : [];
const toggle = (opt: string) =>
@@ -42,7 +55,7 @@ export function PolicyFieldRow({
: [...selected, opt],
);
return (
<FormField label={t(field.label)}>
<FormField label={t(field.label)} helperText={helper}>
<div className="portal-policies__field-chips">
{(field.options ?? []).map((opt) => (
<Chip
@@ -61,7 +74,7 @@ export function PolicyFieldRow({
if (field.type === "select") {
return (
<FormField label={t(field.label)}>
<FormField label={t(field.label)} helperText={helper}>
<Select
inputSize="sm"
value={typeof value === "string" ? value : ""}
@@ -76,7 +89,7 @@ export function PolicyFieldRow({
}
return (
<FormField label={t(field.label)}>
<FormField label={t(field.label)} helperText={helper}>
<Input
inputSize="sm"
value={typeof value === "string" ? value : ""}
@@ -85,3 +98,69 @@ export function PolicyFieldRow({
</FormField>
);
}
/** Free-text multi-value entry (Enter or comma adds), for sets that can't be enumerated ahead of
* time — a team's own email domains, say. */
function PolicyTagsField({
value,
placeholder,
onChange,
id,
"aria-describedby": describedBy,
}: {
value: string[];
placeholder?: string;
onChange: (next: string[]) => void;
/** FormField hands these to its child; they belong on the input the label points at. */
id?: string;
"aria-describedby"?: string;
}) {
const { t } = useTranslation();
const [draft, setDraft] = useState("");
function commit() {
const entry = draft.trim().replace(/^@/, "").toLowerCase();
setDraft("");
if (entry && !value.includes(entry)) onChange([...value, entry]);
}
function onKeyDown(event: KeyboardEvent<HTMLInputElement>) {
if (event.key === "Enter" || event.key === ",") {
// Enter would otherwise submit the wizard, and a comma would land in the
// value rather than separating two of them.
event.preventDefault();
commit();
} else if (event.key === "Backspace" && !draft && value.length > 0) {
onChange(value.slice(0, -1));
}
}
return (
<div className="portal-policies__tags">
{value.length > 0 && (
<div className="portal-policies__field-chips">
{value.map((entry) => (
<Chip
key={entry}
size="sm"
removeLabel={t("common.remove", "Remove") + " " + entry}
onRemove={() => onChange(value.filter((v) => v !== entry))}
>
{entry}
</Chip>
))}
</div>
)}
<Input
id={id}
aria-describedby={describedBy}
inputSize="sm"
value={draft}
placeholder={placeholder}
onChange={(e) => setDraft(e.target.value)}
onKeyDown={onKeyDown}
onBlur={commit}
/>
</div>
);
}
@@ -11,6 +11,7 @@ const security = POLICY_CATEGORIES.find((c) => c.id === "security")!;
const classification = POLICY_CATEGORIES.find(
(c) => c.id === "classification",
)!;
const sharing = POLICY_CATEGORIES.find((c) => c.id === "sharing")!;
const meta: Meta<typeof PolicySetupWizard> = {
title: "Portal/Policies/PolicySetupWizard",
@@ -61,3 +62,11 @@ export const Classification: Story = {
},
},
};
/** Sharing: an egress policy. "Runs on" offers share channels instead of input sources, and the
* editor-only output naming is gone — the copy is replaced in flight. */
export const Sharing: Story = {
args: {
entry: { category: sharing, config: POLICY_CONFIG.sharing, policy: null },
},
};
@@ -5,6 +5,9 @@ import EditOutlinedIcon from "@mui/icons-material/EditOutlined";
import FolderOutlinedIcon from "@mui/icons-material/FolderOutlined";
import CloudOutlinedIcon from "@mui/icons-material/CloudOutlined";
import StorageOutlinedIcon from "@mui/icons-material/StorageOutlined";
import PeopleOutlinedIcon from "@mui/icons-material/PeopleOutlined";
import LinkOutlinedIcon from "@mui/icons-material/LinkOutlined";
import MailOutlinedIcon from "@mui/icons-material/MailOutlined";
import {
Banner,
Button,
@@ -19,6 +22,7 @@ import {
import { SettingsRow } from "@app/ui/SettingsRow";
import {
humanizeEndpoint,
SHARE_CHANNELS,
type CatalogueEntry,
type PipelineStep,
type PolicySetupResult,
@@ -59,6 +63,21 @@ function sourceIcon(type: string): ReactNode {
}
}
/** Outline icon for a share-channel tile, so it reads like the source tiles. */
function channelIcon(id: string): ReactNode {
const sx = { fontSize: "1.1rem" } as const;
switch (id) {
case "userShare":
return <PeopleOutlinedIcon sx={sx} />;
case "shareLink":
return <LinkOutlinedIcon sx={sx} />;
case "emailShare":
return <MailOutlinedIcon sx={sx} />;
default:
return <StorageOutlinedIcon sx={sx} />;
}
}
interface PolicySetupWizardProps {
/** The category being configured, or null when closed. */
entry: CatalogueEntry | null;
@@ -90,9 +109,8 @@ function resolveFieldValues(
* round-trips); otherwise the category preset's default chain. Each preset step
* starts enabled — the user toggles tools off in the workflow.
*/
// Temporary until the catalogue carries a defaultEnabled flag.
// Steps that cannot work until someone configures them, so they start off rather than failing
// every run of a freshly created policy. Purview needs a tenant connection and a label GUID.
// Steps that cannot work until configured, so they start off. A preset that DOES supply the
// parameters overrides this with its own `defaultOn`.
const DISABLED_BY_DEFAULT = new Set<PolicyToolId>([
"watermark",
"purviewApplyLabel",
@@ -202,6 +220,7 @@ function seedTools(entry: CatalogueEntry): ToolState[] {
const step = policyStepFromWire(wire);
if (step) savedByTool.set(step.toolId, step);
}
const defaultOn = entry.config.defaultOn;
// defaultOperations is the canonical list (so tools added later still show on edit); a saved
// step's params win over the preset.
return entry.config.defaultOperations.map((preset) => {
@@ -212,7 +231,9 @@ function seedTools(entry: CatalogueEntry): ToolState[] {
? true
: savedSteps.length > 0
? false
: !DISABLED_BY_DEFAULT.has(preset.toolId),
: defaultOn
? defaultOn.includes(preset.toolId)
: !DISABLED_BY_DEFAULT.has(preset.toolId),
};
});
}
@@ -254,6 +275,9 @@ function PolicySetupWizardBody({
const { category, config, policy } = entry;
const isEdit = policy != null;
const isClassification = category.id === "classification";
// Runs on shares, not the editor: share channels instead of sources, no output naming, and
// gating access with no tool chain is a valid configuration.
const isEgress = category.runsAtEgress === true;
const [step, setStep] = useState<Step>("workflow");
const [tools, setTools] = useState<ToolState[]>(() => {
@@ -268,8 +292,18 @@ function PolicySetupWizardBody({
const [fieldValues, setFieldValues] = useState(() =>
resolveFieldValues(entry),
);
const [sources, setSources] = useState<string[]>(
policy?.state.sources ?? ["editor"],
// Real sources only; editor participation is its own flag, not an entry here.
// For an egress policy these are share channels, not input sources; empty means
// every channel, so a new one starts unnarrowed.
const [sources, setSources] = useState<string[]>(() =>
isEgress
? (policy?.state.sources ?? [])
: (policy?.state.sources ?? []).filter((s) => s !== "editor"),
);
// Whether the policy runs in the editor. Defaults on for a new policy (the common case);
// on edit it comes straight from the stored flag, never re-derived from the sources list.
const [runsOnEditor, setRunsOnEditor] = useState<boolean>(
policy?.state.runsOnEditor ?? true,
);
const sourcesAsync = useSources();
@@ -361,6 +395,12 @@ function PolicySetupWizardBody({
}
function toggleSource(id: string) {
// The editor is not a real source: its tile toggles the runsOnEditor flag instead of
// adding "editor" to the sources list.
if (id === "editor") {
setRunsOnEditor((on) => !on);
return;
}
setSources((prev) =>
prev.includes(id) ? prev.filter((s) => s !== id) : [...prev, id],
);
@@ -368,7 +408,9 @@ function PolicySetupWizardBody({
async function submit() {
if (submitting) return;
if (enabledTools.length === 0) {
// An egress policy that only gates access — no watermark, no redaction — is
// a complete policy, so the at-least-one-tool rule doesn't apply to it.
if (enabledTools.length === 0 && !isEgress) {
setError(t("portal.policies.wizard.errors.noTools"));
setStep("workflow");
return;
@@ -382,6 +424,7 @@ function PolicySetupWizardBody({
await onSubmit(entry, {
fieldValues,
sources,
runsOnEditor,
scopeTypes,
reviewerEmail,
outputMode,
@@ -493,10 +536,15 @@ function PolicySetupWizardBody({
{step === "workflow" && !isClassification && (
<div className="portal-policies__wizard-section">
<p className="portal-policies__wizard-desc">
{t(
"portal.policies.wizard.workflow.description",
"Choose what this policy does to every document it processes.",
)}
{isEgress
? t(
"portal.policies.wizard.workflow.egressDescription",
"Choose what happens to the copy that leaves. Recipients get the processed copy; the original is untouched. Leave these off to govern access only.",
)
: t(
"portal.policies.wizard.workflow.description",
"Choose what this policy does to every document it processes.",
)}
</p>
<Card padding="none">
<div className="portal-policies__capabilities">
@@ -566,6 +614,53 @@ function PolicySetupWizardBody({
{step === "settings" && (
<div className="portal-policies__wizard-section">
{/* Egress policies lead with what they govern, then the terms — the
order the policy is described in. */}
{isEgress && (
<>
<h3 className="portal-policies__wizard-heading">
{t("portal.policies.wizard.channels.heading")}
</h3>
<p className="portal-policies__wizard-desc">
{sources.length === 0
? t("portal.policies.wizard.channels.all")
: t("portal.policies.wizard.channels.narrowed")}
</p>
<div className="portal-policies__sources">
{SHARE_CHANNELS.map((channel) => {
const on = sources.includes(channel.id);
return (
<Button
key={channel.id}
variant={on ? "secondary" : "quiet"}
justify="between"
fullWidth
className={
"portal-policies__source" +
(on ? " portal-policies__source--on" : "")
}
rightSection={
<CheckIcon
sx={{
fontSize: "1.1rem",
visibility: on ? "visible" : "hidden",
}}
/>
}
onClick={() => toggleSource(channel.id)}
aria-pressed={on}
>
<span className="portal-policies__source-label">
{channelIcon(channel.id)}
{t(channel.label)}
</span>
</Button>
);
})}
</div>
</>
)}
{config.fields.length > 0 && (
<>
<h3 className="portal-policies__wizard-heading">
@@ -586,57 +681,72 @@ function PolicySetupWizardBody({
</>
)}
<h3 className="portal-policies__wizard-heading">
{t("portal.policies.wizard.sources.heading")}
</h3>
{sourcesAsync.loading && !sourcesAsync.data ? (
<p className="portal-policies__sources-loading">
{t("portal.policies.wizard.sources.loading")}
</p>
) : (
// The backend always returns the editor as a virtual source, so the
// loaded list is never empty - no "no sources" state exists.
<div className="portal-policies__sources">
{availableSources.map((src) => {
const on = sources.includes(src.id);
return (
<Button
key={src.id}
variant={on ? "secondary" : "quiet"}
justify="between"
fullWidth
className={
"portal-policies__source" +
(on ? " portal-policies__source--on" : "")
}
// The check keeps its slot when unselected (hidden) so the
// icon + name stay put whether or not the tile is selected.
rightSection={
<CheckIcon
sx={{
fontSize: "1.1rem",
visibility: on ? "visible" : "hidden",
}}
/>
}
onClick={() => toggleSource(src.id)}
aria-pressed={on}
>
<span className="portal-policies__source-label">
{sourceIcon(src.type)}
{src.name}
</span>
</Button>
);
})}
</div>
{/* Input sources are for policies that pull files. An egress policy
acts on documents already in storage, so it has none — its
equivalent is the share channels above. */}
{!isEgress && (
<h3 className="portal-policies__wizard-heading">
{t("portal.policies.wizard.sources.heading")}
</h3>
)}
{!isEgress &&
(sourcesAsync.loading && !sourcesAsync.data ? (
<p className="portal-policies__sources-loading">
{t("portal.policies.wizard.sources.loading")}
</p>
) : (
// The backend always returns the editor as a virtual source, so the
// loaded list is never empty - no "no sources" state exists.
<div className="portal-policies__sources">
{availableSources.map((src) => {
// The editor tile toggles runsOnEditor; real sources toggle membership.
const on =
src.id === "editor"
? runsOnEditor
: sources.includes(src.id);
return (
<Button
key={src.id}
variant={on ? "secondary" : "quiet"}
justify="between"
fullWidth
className={
"portal-policies__source" +
(on ? " portal-policies__source--on" : "")
}
// The check keeps its slot when unselected (hidden) so the
// icon + name stay put whether or not the tile is selected.
rightSection={
<CheckIcon
sx={{
fontSize: "1.1rem",
visibility: on ? "visible" : "hidden",
}}
/>
}
onClick={() => toggleSource(src.id)}
aria-pressed={on}
>
<span className="portal-policies__source-label">
{sourceIcon(src.type)}
{src.name}
</span>
</Button>
);
})}
</div>
))}
<h3 className="portal-policies__wizard-heading">
{t("portal.policies.wizard.output.heading")}
</h3>
{/* Output naming applies to a run that produces a file in the editor.
An egress policy replaces the copy in flight, so there is nothing
to name. */}
{!isEgress && (
<h3 className="portal-policies__wizard-heading">
{t("portal.policies.wizard.output.heading")}
</h3>
)}
<div className="portal-policies__fields">
{sources.includes("editor") && (
{!isEgress && runsOnEditor && (
<>
<FormField
label={t("portal.policies.wizard.output.runOn.label")}
@@ -5,6 +5,7 @@
*/
import { fromWirePolicy } from "@app/policies/codec";
import { runsToActivity, runsToStats } from "@app/policies/runs";
import { policyStepToWire } from "@app/policies/operations";
import {
POLICY_CATEGORIES,
POLICY_CONFIG,
@@ -52,3 +53,34 @@ export function decorateForStory(categoryId: string): DecoratedPolicy {
activity: runsToActivity(policyRuns),
};
}
/** A configured Sharing policy: like {@link decorateForStory} but sharing-shaped, since the seeded
* mock policies are all upload/export ones. */
export function decorateSharingForStory(
overrides: Partial<PolicyState> = {},
): DecoratedPolicy {
const base = decorateForStory("sharing");
return {
...base,
steps: [
policyStepToWire(
POLICY_CONFIG.sharing.defaultOperations.find(
(s) => s.toolId === "watermark",
)!,
),
],
state: {
...base.state,
sources: [],
scopeTypes: [],
fieldValues: {
defaultAccess: "restricted",
externalRecipients: "restrict",
internalDomains: ["example.com"],
linkExpiry: "sevenDays",
downloads: "allow",
},
...overrides,
},
};
}
@@ -194,6 +194,18 @@
color: var(--c-text-subtle);
}
/* A field label that carries an info icon explaining the choice. */
.portal-builder__label-hint {
display: inline-flex;
align-items: center;
gap: 0.25rem;
cursor: help;
}
.portal-builder__label-hint svg {
color: var(--c-text-subtle);
}
.portal-builder__input-row .portal-builder__source-edit:hover:not(:disabled) {
color: var(--c-accent-fg, var(--c-primary));
}
@@ -270,6 +270,20 @@ const POLICY: Policy = {
outputIds: [],
};
/** The built-in editor source, offered as an input so a pipeline can run in the browser. */
const EDITOR_SOURCE: SourceView = {
id: "src-editor",
name: "Editor",
type: "editor",
status: "active",
referenceCount: 0,
referencingPolicies: [],
config: [],
docsTotal: 0,
docs24h: 0,
docs30d: 0,
};
const SOURCE: SourceView = {
id: "src-in",
name: "Claims intake",
@@ -799,6 +813,31 @@ describe("PipelineBuilder", () => {
expect(screen.getByText("source-modal:src-1")).toBeInTheDocument();
});
it("saves an editor pipeline as its own flag, not as a wire input", async () => {
fetchSources.mockResolvedValue({
kpis: [],
sources: [SOURCE, EDITOR_SOURCE],
});
renderBuilder("/processor/pipelines/new");
fireEvent.change(
await screen.findByLabelText("portal.pipelines.composer.name"),
{ target: { value: "Label on upload" } },
);
await addTool("Compress");
await pickInputSource("Editor");
fireEvent.click(screen.getByText("portal.pipelines.composer.create"));
await waitFor(() => expect(savePipeline).toHaveBeenCalledTimes(1));
const body = savePipeline.mock.calls[0][0];
// The editor is virtual: nothing sweeps it server-side, so it is recorded as the policy's own
// editor flag rather than as an input the backend would try to pull from.
expect(body.inputs).toEqual([]);
expect(body.editor).toEqual({ allowed: true, runOn: "upload" });
// And it needs no destination - results land back in the workspace the file came from.
expect(body.outputIds).toEqual([]);
});
it("runs an existing pipeline and reports success", async () => {
renderBuilder("/processor/pipelines/plc-1");
@@ -10,7 +10,6 @@ import {
Banner,
Button,
FormField,
Input,
Modal,
Select,
Spinner,
@@ -103,20 +102,16 @@ import {
newIntegrationStep,
stepOperation,
} from "@portal/components/pipelines/integrationStep";
import {
MANUAL,
MANUAL_OPTION,
PipelineInputTrigger,
type EditorRunOn,
type ScheduleUnit,
type WorkingInput,
} from "@portal/components/pipelines/PipelineInputTrigger";
import "@portal/views/PipelineBuilder.css";
type ScheduleUnit = "MINUTES" | "HOURS" | "DAYS";
const SCHEDULE_UNITS: ScheduleUnit[] = ["MINUTES", "HOURS", "DAYS"];
/** Empty trigger type = manual-only (no automatic trigger). */
const MANUAL = "";
/**
* Sentinel value for the manual choice in the trigger dropdown. Mantine's Select treats an empty
* string as "no selection" (it shows the placeholder, not the option), so the manual option needs a
* real value; it maps to/from the empty {@link MANUAL} trigger type at the edges.
*/
const MANUAL_OPTION = "manual";
const TERMINAL_STATUSES = new Set(["COMPLETED", "FAILED", "CANCELLED"]);
const POLL_INTERVAL_MS = 1500;
const POLL_ATTEMPTS = 60;
@@ -149,14 +144,6 @@ function parseTrigger(trigger: TriggerConfig | null): {
return { triggerType: trigger.type, count: "1", unit: "HOURS" };
}
/** One input row in the builder: a source paired with its own trigger config. */
interface WorkingInput {
sourceId: string;
triggerType: string;
scheduleCount: string;
scheduleUnit: ScheduleUnit;
}
/** The input row with nothing chosen yet: no source, manual trigger. */
function blankInput(): WorkingInput {
return {
@@ -237,13 +224,10 @@ export function PipelineBuilder() {
async () => await fetchTriggers(),
[],
);
// The editor is a built-in, client-driven source (it runs on editor upload,
// not as a pipeline input), so it's excluded from a pipeline's inputs.
// Includes the virtual editor source: a valid input, but never a wire input (see save) and not
// writable, so isWritableSource keeps it out of the destinations below.
const availableSources = useMemo<SourceView[]>(
() =>
(sourcesState.data?.sources ?? []).filter(
(source) => source.type !== EDITOR_SOURCE_TYPE,
),
() => sourcesState.data?.sources ?? [],
[sourcesState.data],
);
// A destination is a source used as a write target: only writable types (folder/S3, filtered per
@@ -262,6 +246,17 @@ export function PipelineBuilder() {
// Exactly one input: the row is always present, so the working state is a single object; the
// wire shape stays a list (see save()).
const [input, setInput] = useState<WorkingInput>(blankInput);
// When the editor is the source, the pipeline fires client-side on each file: on upload as it
// arrives, or on export as it leaves. Meaningless for a swept source, which has no such moment.
const [runOn, setRunOn] = useState<EditorRunOn>("upload");
const isEditorInput = useMemo(
() =>
availableSources.some(
(source) =>
source.id === input.sourceId && source.type === EDITOR_SOURCE_TYPE,
),
[availableSources, input.sourceId],
);
const [steps, setSteps] = useState<WorkingToolStep[]>([]);
/** Which node the inspector is editing: an end of the chain, a step, or nothing. */
const [selected, setSelected] = useState<GraphSelection>(null);
@@ -345,13 +340,27 @@ export function PipelineBuilder() {
if (seeded) return;
if (isEdit && !policyState.data) return;
const policy = policyState.data ?? undefined;
// An editor pipeline is recognised by the editor source id, which arrives with the sources
// fetch. If the policy loads first, seeding now would latch a blank input and re-save the
// pipeline off the editor (editor.allowed:false), so wait for that fetch to settle.
if (policy?.editor?.allowed && !sourcesState.data && !sourcesState.error) {
return;
}
setName(policy?.name ?? "");
setEnabled(policy?.enabled ?? true);
// The one input row is always present: blank for a new pipeline (or a legacy policy saved
// without inputs), the stored input for an edit. A legacy multi-input policy shows only its
// first input; saving persists just that one (the backend rejects more anyway).
// An editor pipeline has no wire input; it is recognised by its recorded sources.
const editorSourceId = (sourcesState.data?.sources ?? []).find(
(source) => source.type === EDITOR_SOURCE_TYPE,
)?.id;
setRunOn(policy?.editor?.runOn === "export" ? "export" : "upload");
const stored = policy?.inputs[0];
if (stored) {
const seedsEditor = Boolean(policy?.editor?.allowed && editorSourceId);
if (seedsEditor && editorSourceId) {
setInput({ ...blankInput(), sourceId: editorSourceId });
} else if (stored) {
const trigger = parseTrigger(stored.trigger);
setInput({
sourceId: stored.sourceId,
@@ -365,9 +374,16 @@ export function PipelineBuilder() {
setSteps(
(policy?.steps ?? []).map((step) => deserializeToolStep(step, allTools)),
);
setOutputIds(policy?.outputIds ?? []);
setOutputIds(seedsEditor ? [] : (policy?.outputIds ?? []));
setSeeded(true);
}, [isEdit, policyState.data, allTools, seeded]);
}, [
isEdit,
policyState.data,
allTools,
seeded,
sourcesState.data,
sourcesState.error,
]);
const sourceType = (sourceId: string) =>
availableSources.find((s) => s.id === sourceId)?.type;
@@ -413,8 +429,8 @@ export function PipelineBuilder() {
// Changing the source may make the current trigger incompatible (folder-watch on a non-folder);
// drop it back to manual when that happens so the row can't hold an invalid pairing.
function changeInputSource(sourceId: string) {
const type = sourceType(sourceId);
setInput((current) => {
const type = sourceType(sourceId);
const trigger = triggers.find((tr) => tr.type === current.triggerType);
const keepTrigger =
current.triggerType === MANUAL ||
@@ -425,6 +441,11 @@ export function PipelineBuilder() {
triggerType: keepTrigger ? current.triggerType : MANUAL,
};
});
// The editor hands results back to the workspace, so it has no destination to choose.
if (type === EDITOR_SOURCE_TYPE) {
setOutputIds([]);
setOutputAsked(false);
}
}
/** Put an end on the chain and open it, so the click that asks for it also offers the choice. */
@@ -667,10 +688,14 @@ export function PipelineBuilder() {
// Each validity condition is defined exactly once here, then consumed both by the graph (which
// flags each end) and by the blocker list below.
const sourceChosen = input.sourceId !== "";
// An editor pipeline has no trigger to schedule: it fires as each file passes through.
const scheduleValid =
input.triggerType !== "schedule" || Number(input.scheduleCount) > 0;
isEditorInput ||
input.triggerType !== "schedule" ||
Number(input.scheduleCount) > 0;
const inputValid = sourceChosen && scheduleValid;
const outputValid = outputIds.length === 1;
// Nor a destination: an editor pipeline's results land back in the workspace the file came from.
const outputValid = isEditorInput || outputIds.length === 1;
// The single source of truth for "can this be committed": every reason it can't be, in the order
// they appear down the form, so a disabled Create / Save button can say exactly what is still owed.
@@ -781,13 +806,19 @@ export function PipelineBuilder() {
id: policyState.data?.id ?? undefined,
name: name.trim(),
enabled: enabledOverride ?? enabled,
// The wire shape stays a list; canSave guarantees the one input has a source.
inputs: [{ sourceId: input.sourceId, trigger: buildTriggerFor(input) }],
// The editor is virtual - there is no stored Source to pull from, and nothing server-side
// sweeps it - so it is never a wire input; its participation is recorded on `editor` below.
inputs: isEditorInput
? []
: [{ sourceId: input.sourceId, trigger: buildTriggerFor(input) }],
steps: await serializeStepsForSave(),
// Destinations are the referenced saved sources; the inline output field is
// preserved as-is (e.g. an editor policy's membership metadata) or defaults to inline.
// Destinations are the referenced saved sources; the inline output is preserved as-is
// or defaults to inline.
output: policyState.data?.output ?? { type: "inline", options: {} },
outputIds,
editor: { allowed: isEditorInput, runOn },
// An editor pipeline delivers back into the workspace. A stored destination would send the
// run to a folder or bucket instead, leaving the editor's copy untouched.
outputIds: isEditorInput ? [] : outputIds,
};
await savePipeline(policy);
await invalidatePipelines();
@@ -1046,6 +1077,11 @@ export function PipelineBuilder() {
/** How this input fires, in a few words, for the input node's summary line. */
function triggerSummary(): string {
// The editor has no trigger to schedule; it fires as each file passes through.
if (isEditorInput)
return runOn === "export"
? t("portal.pipelines.builder.runOnExport", "Every export")
: t("portal.pipelines.builder.runOnUpload", "Every upload");
if (input.triggerType === MANUAL)
return t("portal.pipelines.composer.triggerManual");
if (input.triggerType === "schedule")
@@ -1158,7 +1194,7 @@ export function PipelineBuilder() {
variant="tertiary"
className="portal-builder__source-edit"
aria-label={t("portal.pipelines.composer.editSource")}
disabled={input.sourceId === ""}
disabled={input.sourceId === "" || isEditorInput}
onClick={() =>
setSourceModal({ open: true, sourceId: input.sourceId })
}
@@ -1168,58 +1204,14 @@ export function PipelineBuilder() {
</div>
</FormField>
<FormField label={t("portal.pipelines.builder.inputTrigger")}>
<Select
inputSize="sm"
aria-label={t("portal.pipelines.builder.inputTrigger")}
value={
input.triggerType === MANUAL
? MANUAL_OPTION
: input.triggerType
}
disabled={input.sourceId === ""}
onChange={(value) =>
updateInput({
triggerType:
value && value !== MANUAL_OPTION ? value : MANUAL,
})
}
options={triggerOptionsFor(input.sourceId)}
/>
</FormField>
{input.triggerType === "schedule" && (
<div className="portal-builder__schedule">
<span className="portal-builder__muted">
{t("portal.pipelines.composer.scheduleEvery")}
</span>
<Input
inputSize="sm"
type="number"
min={1}
value={input.scheduleCount}
invalid={Number(input.scheduleCount) <= 0}
onChange={(e) =>
updateInput({ scheduleCount: e.target.value })
}
className="portal-builder__schedule-count"
/>
<Select
inputSize="sm"
value={input.scheduleUnit}
onChange={(value) =>
value &&
updateInput({ scheduleUnit: value as ScheduleUnit })
}
options={SCHEDULE_UNITS.map((unit) => ({
value: unit,
label: t(
`portal.pipelines.composer.unit.${unit.toLowerCase()}`,
),
}))}
/>
</div>
)}
<PipelineInputTrigger
input={input}
onInputChange={updateInput}
triggerOptions={triggerOptionsFor(input.sourceId)}
isEditorInput={isEditorInput}
runOn={runOn}
onRunOnChange={setRunOn}
/>
</>
)}
@@ -1235,6 +1227,17 @@ export function PipelineBuilder() {
);
}
if (selected === "output" && isEditorInput) {
return (
<p className="portal-builder__muted">
{t(
"portal.pipelines.builder.editorDestinationHelp",
"This pipeline runs on the files in your workspace, and its results replace the file it ran on. There is nowhere else to send them.",
)}
</p>
);
}
if (selected === "output") {
return (
<DestinationPicker
@@ -1344,16 +1347,28 @@ export function PipelineBuilder() {
: null
}
output={
outputAsked || outputValid
isEditorInput
? {
label:
chosenDestination?.name ??
t("portal.pipelines.builder.chooseDestination"),
warning: outputValid
? undefined
: t("portal.pipelines.builder.needsDestination"),
label: t(
"portal.pipelines.builder.editorDestination",
"Editor",
),
detail: t(
"portal.pipelines.builder.editorDestinationDetail",
"Replaces the file you ran it on",
),
fixed: true,
}
: null
: outputAsked || outputValid
? {
label:
chosenDestination?.name ??
t("portal.pipelines.builder.chooseDestination"),
warning: outputValid
? undefined
: t("portal.pipelines.builder.needsDestination"),
}
: null
}
steps={graphSteps}
selected={selected}
@@ -248,6 +248,29 @@
display: flex;
flex-wrap: wrap;
gap: 0.375rem;
align-items: center;
}
/* Free-text multi-value entry (e.g. internal email domains): the committed
chips sit above the input they were typed into. */
.portal-policies__tags {
display: flex;
flex-direction: column;
gap: 0.5rem;
}
/* "Applies to" selected document types, then a search box that offers the
classification vocabulary as you type. */
.portal-policies__scope {
display: flex;
flex-direction: column;
gap: 0.5rem;
}
.portal-policies__scope-empty {
margin: 0;
font-size: 0.8125rem;
color: var(--c-text-subtle);
}
/* Workflow step policy capability settings list.
@@ -1,5 +1,5 @@
import { usePolicyAutoRun } from "@app/components/policies/usePolicyAutoRun";
import { useClientSideClassification } from "@app/components/policies/useClientSideClassification";
import { usePolicyLocalPasses } from "@app/components/policies/usePolicyLocalPasses";
/**
* Headless controller that drives policy auto-run (enforce every enabled policy
@@ -7,8 +7,9 @@ import { useClientSideClassification } from "@app/components/policies/useClientS
* regardless of whether the policy panel is visible. Renders nothing.
*/
export function PolicyAutoRunController() {
// Server-dispatched, file-producing policies and their chain.
usePolicyAutoRun();
// Non-AI systems classify uploads in the browser; inert when the AI engine is on.
useClientSideClassification();
// Policies with a browser-side fast path (e.g. classification's heuristic), run generically.
usePolicyLocalPasses();
return null;
}
@@ -0,0 +1,169 @@
/**
* The Classification policy's browser-side fast path, as a {@link LocalPass} the generic local-pass
* engine runs. Everything classification-specific lives here: the heuristic, the label/confidence it
* writes, metering, and the browser-local run it records. The engine only sees the generic result
* (fields to write + whether the AI server run is still needed).
*/
import { fileStorage } from "@app/services/fileStorage";
import { classifyFileHeuristically } from "@app/services/heuristic/heuristicClassification";
import { meterClassificationRun } from "@app/services/classificationMeter";
import {
isDispatched,
markDispatched,
recordRunStart,
updateRun,
} from "@app/components/policies/policyRunStore";
import type { FileId } from "@app/types/file";
import type { StirlingFile } from "@app/types/fileContext";
import type { HeuristicConfidence } from "@app/services/heuristic/types";
import {
CLASSIFICATION_CATEGORY_ID,
localVerdictNeedsEscalation,
} from "@app/data/classificationPolicy";
import type { LocalPass } from "@app/components/policies/policyLocalPass";
/** How long to wait for an upload's bytes to land in IndexedDB (20 × 250ms 5s).
* The stub can surface in the file list a beat before its bytes are committed. */
const FILE_WAIT_TRIES = 20;
const FILE_WAIT_MS = 250;
/** localStorage flag: set to "true" for a full per-file scoring breakdown in the console. */
const DEBUG_FLAG = "stirling-classification-debug";
function isClassificationDebug(): boolean {
try {
return localStorage.getItem(DEBUG_FLAG) === "true";
} catch {
return false;
}
}
const delay = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));
export const classificationLocalPass: LocalPass = {
// A new document that has not been classified yet. Tool outputs inherit their input's verdict via
// the file reducer, so they are never classified afresh here.
eligible: (stub) =>
!stub.derivedFromTool && stub.classificationLabels == null,
run: async (fileId, stub) => {
const verdict = await classifyStub(fileId, stub.name, stub.size ?? 0);
// Bytes never landed (file removed mid-wait): leave unclassified so a reload retries.
if (verdict == null) return null;
return {
stubUpdates: {
classificationLabels: verdict.labels,
classificationConfidence: verdict.confidence,
},
// A confident local verdict stands; anything less asks the AI engine, which overwrites it.
needsServerRun: localVerdictNeedsEscalation(verdict.confidence),
};
},
};
/** Classify one file, metering exactly once; null = no verdict, retried later. */
async function classifyStub(
fileId: FileId,
fileName: string,
fileSize: number,
): Promise<{ labels: string[]; confidence: HeuristicConfidence } | null> {
let file: StirlingFile | null = null;
for (let i = 0; i < FILE_WAIT_TRIES; i++) {
file = await fileStorage.getStirlingFile(fileId).catch(() => null);
if (file) break;
await delay(FILE_WAIT_MS);
}
if (!file) {
console.warn(
`[Classify] ${fileName}: bytes never arrived in storage; will retry on next load`,
);
return null;
}
const debug = isClassificationDebug();
const startedAt = performance.now();
// A local run is still a billable policy run, so it belongs in the activity feed; recorded only
// once the bytes are in hand, so a file whose bytes never land leaves no phantom row.
// Read before recordRunStart, which takes the dispatch key itself and would otherwise always
// answer "already dispatched", silently stopping metering.
const alreadyMetered = isDispatched(CLASSIFICATION_CATEGORY_ID, fileId);
const runId = `local-${CLASSIFICATION_CATEGORY_ID}-${fileId}-${Date.now()}`;
recordRunStart({
runId,
categoryId: CLASSIFICATION_CATEGORY_ID,
fileId: fileId as string,
fileName,
fileSize,
target: "local",
// The heuristic ran in the browser - there is no server run to poll (see the poll effect).
browserLocal: true,
status: "RUNNING",
outputs: [],
error: null,
startedAt: Date.now(),
});
try {
const result = await classifyFileHeuristically(file, { explain: debug });
const { labels } = result;
const ms = Math.round(performance.now() - startedAt);
const verdict =
labels.length > 0
? labels.join(", ")
: result.isEnglish
? "no label"
: "no label (not English)";
console.debug(
`[Classify] ${fileName} -> ${verdict} (${result.confidence}, score ${result.score}, ${ms}ms)` +
(alreadyMetered ? " [heal: not re-metered]" : ""),
);
if (debug && result.explain) logExplanation(fileName, result);
// Meter on the first classification only; a healing re-run of an undelivered
// result (already dispatched) is not a new billable run.
if (!alreadyMetered) {
meterClassificationRun({
policyName: "Classification",
documentCount: 1,
labels,
});
}
markDispatched(CLASSIFICATION_CATEGORY_ID, fileId);
// Labels, no output file - the same settle shape the server-run classification uses.
updateRun(runId, {
status: "COMPLETED",
imported: true,
outputFileIds: [fileId as string],
});
return { labels, confidence: result.confidence };
} catch (err) {
// Never persist a verdict for an unreadable file - the failure may be
// environmental, so it must stay eligible to retry (and meter) later.
console.warn(`[Classify] ${fileName}: could not be read, will retry`, err);
updateRun(runId, {
status: "FAILED",
error: err instanceof Error ? err.message : String(err),
});
return null;
}
}
/** Full scoring breakdown, one collapsed console group per file (debug flag only). */
function logExplanation(
fileName: string,
result: Awaited<ReturnType<typeof classifyFileHeuristically>>,
): void {
const ex = result.explain;
if (!ex) return;
console.groupCollapsed(
`[Classify] ${fileName} scoring (english=${ex.isEnglish}, lowText=${ex.lowText})`,
);
if (ex.candidates.length === 0) {
console.log("no label scored above zero");
}
for (const c of ex.candidates) {
console.log(
`${c.id}${c.emit ? "" : " (suppressed)"}: score ${c.score}, ${c.distinct} distinct signals`,
);
for (const s of c.signals) console.log(` ${s}`);
}
console.groupEnd();
}
@@ -0,0 +1,34 @@
/**
* The generic "browser-side fast path" seam. A policy may declare a {@link LocalPass}: cheap local
* work that runs before any server dispatch and can settle a file on its own, or decide the server
* run is still needed. The local-pass engine ({@link ../../hooks/usePolicyLocalPasses}) runs it
* without knowing what it computes; the policy-specific logic lives entirely inside the pass.
*/
import type { FileId } from "@app/types/file";
import type { StirlingFileStub } from "@app/types/fileContext";
import { CLASSIFICATION_CATEGORY_ID } from "@app/data/classificationPolicy";
import { classificationLocalPass } from "@app/components/policies/classificationLocalPass";
export interface LocalPassResult {
/** Fields to merge onto the file's stub and stored metadata. Opaque to the engine. */
stubUpdates: Partial<StirlingFileStub>;
/** Whether the policy's server run should still be dispatched after this pass. */
needsServerRun: boolean;
}
export interface LocalPass {
/** Files this pass should run on (e.g. new documents it has not processed yet). */
eligible(stub: StirlingFileStub): boolean;
/**
* Do the local work for one file. Returns the stub fields to write and whether the server run is
* still needed, or null if the work could not be done and should be retried later.
*/
run(fileId: FileId, stub: StirlingFileStub): Promise<LocalPassResult | null>;
}
/** The local fast path a policy declares, if any. The default (most policies) is none. */
export function localPassFor(categoryId: string): LocalPass | undefined {
if (categoryId === CLASSIFICATION_CATEGORY_ID) return classificationLocalPass;
return undefined;
}
@@ -83,29 +83,6 @@ describe("policyRunStore", () => {
expect(isDispatched("security", "f1")).toBe(true);
});
it("a browser-local run does not claim the (policy, file) dispatch key", () => {
// The local classification heuristic records a run for the same (classification, file) pair
// the server escalation is keyed on. If that claimed the key, the auto-run would read
// "already dispatched" and never ask the AI - which killed escalation entirely.
recordRunStart(
rec({
runId: "local-classification-f1-1",
categoryId: "classification",
fileId: "f1",
target: "local",
browserLocal: true,
status: "RUNNING",
}),
);
expect(getRun("local-classification-f1-1")).toBeDefined();
expect(isDispatched("classification", "f1")).toBe(false);
});
it("a real backend run still claims the dispatch key", () => {
recordRunStart(rec({ runId: "srv-1", categoryId: "classification" }));
expect(isDispatched("classification", "f1")).toBe(true);
});
it("never evicts in-flight runs, even past the soft cap", () => {
// A large upload batch can exceed the cap while still processing. Dropping a
// live run would orphan its polling/import and undercount progress, so every
@@ -45,16 +45,12 @@ export interface PolicyRunRecord {
/** Set while an auto-retry is pending after a transient (queue-full) rejection, so the activity
* feed shows a soft "busy" row instead of a hard failure during the backoff window. */
retrying?: boolean;
/** A run computed entirely in the browser - it has no server run behind it,
* so it must never be polled for status (a status poll 404s and would flip a
* succeeded run to FAILED) or reconciled against the server. */
browserLocal?: boolean;
/** Epoch ms when the run was dispatched. */
startedAt: number;
/**
* Ran in the browser (the local classification heuristic), not on a backend. Such a run has no
* server-side status to poll, and - crucially - must NOT claim the (policy, file) dispatch key:
* it is the first pass, not the policy's run, so claiming it would suppress the server run the
* verdict may still need to escalate to. Distinct from {@link target}, which says which BACKEND
* holds a real run's outputs.
*/
browserLocal?: boolean;
}
/** Statuses of a run that is still executing (not yet settled). */
@@ -224,15 +220,11 @@ export function recordRunStart(record: PolicyRunRecord) {
const waveStartedAt = state.runs.some(isRunInFlight)
? state.waveStartedAt
: record.startedAt;
// A browser-local run is the first pass, not the policy's run: claiming the dispatch key here
// would permanently suppress the server run its verdict may still need to escalate to.
const claimsDispatch = !record.browserLocal;
state = {
runs: capRuns([record, ...state.runs]),
dispatched:
!claimsDispatch || state.dispatched.includes(key)
? state.dispatched
: [...state.dispatched, key],
dispatched: state.dispatched.includes(key)
? state.dispatched
: [...state.dispatched, key],
waveStartedAt,
};
emit();
@@ -4,6 +4,7 @@ import type { IconBadgeAccent } from "@app/ui/IconBadge";
export const ROW_ACCENT: Record<string, IconBadgeAccent> = {
ingestion: "neutral",
security: "neutral",
sharing: "neutral",
compliance: "neutral",
routing: "neutral",
retention: "neutral",
@@ -16,6 +17,7 @@ const BADGE_ACCENT: Record<string, string> = {
ingestion: "blue",
classification: "orange",
security: "purple",
sharing: "green",
compliance: "green",
routing: "amber",
retention: "red",
@@ -1,251 +0,0 @@
// The Classification policy's first pass: every upload is labelled locally before the AI is asked.
// The confidence reported here decides whether the AI is asked at all - see usePolicyAutoRun.
import { useEffect, useRef, useState } from "react";
import { useAllFiles, useFileManagement } from "@app/contexts/FileContext";
import { useAppConfig } from "@app/contexts/AppConfigContext";
import { useIndexedDB } from "@app/contexts/IndexedDBContext";
import { fileStorage } from "@app/services/fileStorage";
import { useClassificationEnabled } from "@app/hooks/useClassificationEnabled";
import { scheduleIdle } from "@app/utils/scheduleIdle";
import { usePolicies } from "@app/hooks/usePolicies";
import { classifyFileHeuristically } from "@app/services/heuristic/heuristicClassification";
import { meterClassificationRun } from "@app/services/classificationMeter";
import {
isDispatched,
markDispatched,
recordRunStart,
updateRun,
} from "@app/components/policies/policyRunStore";
import type { FileId } from "@app/types/file";
import type { StirlingFile, StirlingFileStub } from "@app/types/fileContext";
import type { HeuristicConfidence } from "@app/services/heuristic/types";
import { CLASSIFICATION_CATEGORY_ID } from "@app/data/classificationPolicy";
/**
* Dispatch-store key namespace for "this file's local pass has been metered". Deliberately NOT the
* Classification category id: that key is the server escalation's own guard, so metering under it
* would tell the auto-run the policy had already run and kill the escalation entirely.
*/
export const LOCAL_METER_CATEGORY = `${CLASSIFICATION_CATEGORY_ID}:local-meter`;
/** Files classified per idle pass, so a large library drains over several ticks. */
const CLASSIFY_BATCH = 3;
/** How long to wait for an upload's bytes to land in IndexedDB (20 × 250ms 5s).
* The stub can surface in the file list a beat before its bytes are committed. */
const FILE_WAIT_TRIES = 20;
const FILE_WAIT_MS = 250;
/** localStorage flag: set to "true" for a full per-file scoring breakdown in the console. */
const DEBUG_FLAG = "stirling-classification-debug";
function isClassificationDebug(): boolean {
try {
return localStorage.getItem(DEBUG_FLAG) === "true";
} catch {
return false;
}
}
const delay = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));
export function useClientSideClassification(): void {
const { fileStubs } = useAllFiles();
const { updateStirlingFileStub } = useFileManagement();
const { bumpRevision } = useIndexedDB();
const { policies } = usePolicies();
const classificationEnabled = useClassificationEnabled();
// Still waited on: a verdict written before app-config lands would be acted on by the
// escalation decision before it knows whether the AI engine is even available.
const { loading: configLoading } = useAppConfig();
// Files claimed this session, keyed id+lastModified so a new version is retried once. A claim is
// taken synchronously right before classifying, so overlapping batches never double-classify.
const claimed = useRef<Set<string>>(new Set());
// Bumped after each batch to drain the next one.
const [tick, setTick] = useState(0);
// TODO: keyed on the Classification CATEGORY, so a pipeline that merely contains a classify
// step gets no local pass - suppressing one step of a chain is not expressible today.
const policy = policies[CLASSIFICATION_CATEGORY_ID];
// Only when the admin has an active Classification policy - the same gate the AI path uses.
const active = Boolean(
policy?.configured &&
policy.status === "active" &&
policy.backendId &&
(!policy.sources ||
policy.sources.length === 0 ||
policy.sources.includes("editor")),
);
useEffect(() => {
// Runs whether or not the AI engine is on: it is the first pass either way, not a fallback.
if (configLoading || !classificationEnabled || !active) {
return;
}
const claimKey = (s: StirlingFileStub) =>
`${s.id as string}:${s.lastModified ?? 0}`;
// null labels = never delivered, retried here; [] = definitive no-label verdict.
const pending = fileStubs
.filter(
(s) =>
!s.derivedFromTool &&
s.classificationLabels == null &&
!claimed.current.has(claimKey(s)),
)
.slice(0, CLASSIFY_BATCH);
if (pending.length === 0) return;
let cancelled = false;
const cancelIdle = scheduleIdle(() => {
// Superseded before starting: the newer effect instance owns the queue.
if (cancelled) return;
void (async () => {
let wrote = false;
for (const stub of pending) {
const key = claimKey(stub);
// Re-validate at execution time - another batch may have claimed it since.
if (claimed.current.has(key)) continue;
claimed.current.add(key);
const verdict = await classifyStub(
stub.id,
stub.name,
stub.size ?? 0,
);
// Bytes never landed (file removed mid-wait): leave undelivered so a
// reload (or new version) retries; the claim stops churn this session.
if (verdict == null) continue;
// Deliver unconditionally - a re-render must never discard a computed
// (and already metered) result. Writes are idempotent.
updateStirlingFileStub(stub.id, {
classificationLabels: verdict.labels,
classificationConfidence: verdict.confidence,
});
const ok = await fileStorage.updateFileMetadata(stub.id, {
classificationLabels: verdict.labels,
classificationConfidence: verdict.confidence,
});
if (ok) wrote = true;
}
if (wrote) bumpRevision();
// Drain the next batch; the terminal pass finds nothing pending and stops.
setTick((n) => n + 1);
})();
});
return () => {
cancelled = true;
cancelIdle();
};
}, [
fileStubs,
active,
classificationEnabled,
configLoading,
updateStirlingFileStub,
bumpRevision,
tick,
]);
}
/** Classify one file, metering exactly once; null = no verdict, retried later. */
async function classifyStub(
fileId: FileId,
fileName: string,
fileSize: number,
): Promise<{ labels: string[]; confidence: HeuristicConfidence } | null> {
let file: StirlingFile | null = null;
for (let i = 0; i < FILE_WAIT_TRIES; i++) {
file = await fileStorage.getStirlingFile(fileId).catch(() => null);
if (file) break;
await delay(FILE_WAIT_MS);
}
if (!file) {
console.warn(
`[Classify] ${fileName}: bytes never arrived in storage; will retry on next load`,
);
return null;
}
const debug = isClassificationDebug();
const startedAt = performance.now();
// A local run is still a billable policy run, so it belongs in the activity feed; recorded only
// once the bytes are in hand, so a file whose bytes never land leaves no phantom row.
const alreadyMetered = isDispatched(LOCAL_METER_CATEGORY, fileId);
const runId = `local-${CLASSIFICATION_CATEGORY_ID}-${fileId}-${Date.now()}`;
recordRunStart({
runId,
categoryId: CLASSIFICATION_CATEGORY_ID,
fileId: fileId as string,
fileName,
fileSize,
target: "local",
// Ran here, not on a backend: nothing to poll, and it must not claim the classification
// dispatch key - that key is what the server escalation checks before running.
browserLocal: true,
status: "RUNNING",
outputs: [],
error: null,
startedAt: Date.now(),
});
try {
const result = await classifyFileHeuristically(file, { explain: debug });
const { labels } = result;
const ms = Math.round(performance.now() - startedAt);
const verdict =
labels.length > 0
? labels.join(", ")
: result.isEnglish
? "no label"
: "no label (not English)";
console.debug(
`[Classify] ${fileName} -> ${verdict} (${result.confidence}, score ${result.score}, ${ms}ms)` +
(alreadyMetered ? " [heal: not re-metered]" : ""),
);
if (debug && result.explain) logExplanation(fileName, result);
// Meter on the first classification only; a healing re-run of an undelivered
// result (already dispatched) is not a new billable run.
if (!alreadyMetered) {
meterClassificationRun({
policyName: "Classification",
documentCount: 1,
labels,
});
}
markDispatched(LOCAL_METER_CATEGORY, fileId);
// Labels, no output file - the same settle shape the server-run classification uses.
updateRun(runId, {
status: "COMPLETED",
imported: true,
outputFileIds: [fileId as string],
});
return { labels, confidence: result.confidence };
} catch (err) {
// Never persist a verdict for an unreadable file - the failure may be
// environmental, so it must stay eligible to retry (and meter) later.
console.warn(`[Classify] ${fileName}: could not be read, will retry`, err);
updateRun(runId, {
status: "FAILED",
error: err instanceof Error ? err.message : String(err),
});
return null;
}
}
/** Full scoring breakdown, one collapsed console group per file (debug flag only). */
function logExplanation(
fileName: string,
result: Awaited<ReturnType<typeof classifyFileHeuristically>>,
): void {
const ex = result.explain;
if (!ex) return;
console.groupCollapsed(
`[Classify] ${fileName} scoring (english=${ex.isEnglish}, lowText=${ex.lowText})`,
);
if (ex.candidates.length === 0) {
console.log("no label scored above zero");
}
for (const c of ex.candidates) {
console.log(
`${c.id}${c.emit ? "" : " (suppressed)"}: score ${c.score}, ${c.distinct} distinct signals`,
);
for (const s of c.signals) console.log(` ${s}`);
}
console.groupEnd();
}
@@ -3,8 +3,10 @@ import { renderHook, act } from "@testing-library/react";
import type { ClassificationConfidence } from "@app/types/fileContext";
/**
* Batch integration test (61 files, two chained upload policies) driving the real
* store + hook effects, IO mocked. Classification is forced last (see the sort).
* Batch integration test (61 files, Security then Classification) driving the real store + both
* hooks, IO mocked. The auto-run engine dispatches the file-producing Security policy and versions
* its output in place; the Classification policy runs itself (useClassificationPolicy) on each settled
* output - here a confident local verdict, so it stamps labels without escalating to the AI.
*/
const FILE_COUNT = 61;
@@ -15,6 +17,9 @@ const FILE_COUNT = 61;
const mocks = vi.hoisted(() => ({
workspace: [] as Array<{
id: string;
name?: string;
size?: number;
derivedFromTool?: boolean;
classificationLabels?: string[];
classificationConfidence?: ClassificationConfidence;
}>,
@@ -39,13 +44,25 @@ const mocks = vi.hoisted(() => ({
addFiles: vi.fn(),
updateStirlingFileStub: vi.fn(),
consumeFiles: vi.fn(),
classify: vi.fn(),
meter: vi.fn(),
}));
// The second (file-producing) policy's timing, flippable per test to prove classification's local
// pass is independent of when the rewriter runs.
const securityRunOn = vi.hoisted(() => ({
value: "upload" as "upload" | "export",
}));
// Classification chains server-side only when the AI engine is on (else it runs
// client-side); this batch exercises the server chain, so force the engine on.
vi.mock("@app/hooks/useAiEngineEnabled", () => ({
useAiEngineEnabled: () => true,
}));
vi.mock("@app/hooks/useClassificationEnabled", () => ({
useClassificationEnabled: () => true,
}));
vi.mock("@app/contexts/AppConfigContext", () => ({
useAppConfig: () => ({ config: {}, loading: false }),
}));
vi.mock("@app/contexts/FileContext", () => ({
useAllFiles: () => ({ fileStubs: mocks.workspace }),
useFileManagement: () => ({
@@ -60,10 +77,9 @@ vi.mock("@app/contexts/IndexedDBContext", () => ({
vi.mock("@app/hooks/usePolicies", () => ({
usePolicies: () => ({
policies: {
// Classification is configured first (order 0) but is FORCED to run last
// by the orchestrator; Security (order 1) therefore runs first.
classification: {
configured: true,
runsOnEditor: true,
status: "active",
backendId: "backend-classification",
runOn: "upload",
@@ -73,9 +89,10 @@ vi.mock("@app/hooks/usePolicies", () => ({
},
security: {
configured: true,
runsOnEditor: true,
status: "active",
backendId: "backend-security",
runOn: "upload",
runOn: securityRunOn.value,
order: 1,
outputMode: "new_version",
outputName: "",
@@ -102,39 +119,54 @@ vi.mock("@app/services/fileStubHelpers", () => ({
createStirlingFilesAndStubs: mocks.createStirlingFilesAndStubs,
}));
vi.mock("@app/services/fileClassification", () => ({
// Classification always resolves labels here, so the metadata-only import path
// stamps them onto the stub.
// The AI import path (if ever reached) resolves labels from the output PDF.
readClassificationLabelsFromFile: vi.fn().mockResolvedValue(["Invoice"]),
}));
vi.mock("@app/services/heuristic/heuristicClassification", () => ({
classifyFileHeuristically: (file: File) => mocks.classify(file),
}));
vi.mock("@app/services/classificationMeter", () => ({
meterClassificationRun: (payload: unknown) => mocks.meter(payload),
}));
import { usePolicyAutoRun } from "@app/components/policies/usePolicyAutoRun";
import { usePolicyLocalPasses } from "@app/components/policies/usePolicyLocalPasses";
import {
usePolicyRuns,
resetPolicyRuns,
} from "@app/components/policies/policyRunStore";
import type { PolicyRunRecord } from "@app/components/policies/policyRunStore";
// Run idle callbacks immediately so the local-pass engine's batches start without timer waits.
vi.stubGlobal("requestIdleCallback", (cb: () => void) => {
cb();
return 1;
});
vi.stubGlobal("cancelIdleCallback", () => {});
/** A stable snapshot of the store, read after the flow settles. */
let latestRuns: PolicyRunRecord[] = [];
function Harness() {
usePolicyAutoRun();
usePolicyLocalPasses();
latestRuns = usePolicyRuns();
return null;
}
/** The heuristic verdict that escalates to the AI classifier; only "high" stands alone. */
const LOW = "low" as const;
function replaceInWorkspace(inputIds: string[], outputIds: string[]) {
// A versioned output carries its input's heuristic verdict; the escalation decision is about the
// document, not about which step produced the current bytes.
const inherited =
mocks.workspace.find((s) => inputIds.includes(s.id))
?.classificationConfidence ?? LOW;
// A versioned output inherits its input's classification verdict, exactly as the real CONSUME_FILES
// reducer does - so a label put on the upload rides forward without re-classifying the output.
const donor = mocks.workspace.find((s) => inputIds.includes(s.id));
mocks.workspace = mocks.workspace
.filter((s) => !inputIds.includes(s.id))
.concat(
outputIds.map((id) => ({ id, classificationConfidence: inherited })),
outputIds.map((id) => ({
id,
name: "doc.pdf",
derivedFromTool: true,
classificationLabels: donor?.classificationLabels,
classificationConfidence: donor?.classificationConfidence,
})),
);
}
@@ -150,10 +182,11 @@ beforeEach(() => {
mocks.backendOutCounter = 0;
mocks.dispatchInFlight = 0;
mocks.maxDispatchInFlight = 0;
securityRunOn.value = "upload";
mocks.workspace = Array.from({ length: FILE_COUNT }, (_, i) => ({
id: `file-${i}`,
classificationConfidence: LOW,
name: `doc-${i}.pdf`,
}));
mocks.listPolicyRuns.mockResolvedValue([]);
@@ -169,8 +202,14 @@ beforeEach(() => {
mocks.downloadPolicyOutput.mockResolvedValue(
new Blob(["x"], { type: "application/pdf" }),
);
// Apply stub updates to the shared workspace, as the real reducer does — the
// label stamp's second pass reads them back to stay idempotent.
// A confident local verdict: classification stamps labels and does NOT escalate to the AI.
mocks.classify.mockResolvedValue({
labels: ["Invoice"],
confidence: "high",
isEnglish: true,
score: 5,
});
// Apply stub updates to the shared workspace, as the real reducer does.
mocks.updateStirlingFileStub.mockImplementation(
(id: string, updates: Record<string, unknown>) => {
const stub = mocks.workspace.find((s) => s.id === id);
@@ -205,7 +244,7 @@ beforeEach(() => {
],
}));
// Deliver a unique workspace child stub per output, derived from the parent so
// the chain's second policy can find + version it.
// classification can find + tag it.
mocks.createStirlingFilesAndStubs.mockImplementation(
async (files: File[], parentStub: { id: string }) => {
const stubs = files.map(() => ({
@@ -237,7 +276,7 @@ beforeEach(() => {
);
});
/** Drive the hook until the store shows the expected number of imported runs. */
/** Drive the hooks until the store shows the expected number of imported runs. */
async function runUntilSettled(expectedRuns: number) {
renderHook(() => Harness());
await act(async () => {
@@ -263,6 +302,8 @@ describe("policy auto-run — 61-file batch through a Security → Classificatio
expect(classification).toHaveLength(FILE_COUNT);
expect(security).toHaveLength(FILE_COUNT);
expect(latestRuns).toHaveLength(FILE_COUNT * 2);
// A confident local verdict stands on its own: no AI dispatch for classification.
expect(classification.every((r) => r.target === "local")).toBe(true);
});
it("bounds concurrent dispatch uploads so polls/downloads keep connections", async () => {
@@ -280,7 +321,10 @@ describe("policy auto-run — 61-file batch through a Security → Classificatio
// Classification never forks a version — it only stamps labels onto the stub.
expect(mocks.updateStirlingFileStub).toHaveBeenCalledTimes(FILE_COUNT);
for (const call of mocks.updateStirlingFileStub.mock.calls) {
expect(call[1]).toEqual({ classificationLabels: ["Invoice"] });
expect(call[1]).toEqual({
classificationLabels: ["Invoice"],
classificationConfidence: "high",
});
}
// Never added as brand-new files either.
expect(mocks.addFilesCalls).toBe(0);
@@ -303,18 +347,45 @@ describe("policy auto-run — 61-file batch through a Security → Classificatio
await act(async () => {
await vi.waitFor(
() => {
const imported = latestRuns.filter((r) => r.imported).length;
expect(imported).toBe(FILE_COUNT * 2);
const security = latestRuns.filter(
(r) => r.categoryId === "security" && r.imported,
);
expect(security).toHaveLength(FILE_COUNT);
},
{ timeout: 8000, interval: 20 },
);
});
// Still fully processed (chain intact), but Security's versions went to
// STORAGE, never re-added to the workbench — the workspace stays empty.
expect(latestRuns).toHaveLength(FILE_COUNT * 2);
// Security's versions went to STORAGE, never re-added to the workbench, so the
// workspace stays empty. (Classification may have tagged the few files still open
// when the workbench was cleared; the point here is the runner does not re-open them.)
expect(mocks.workspace).toHaveLength(0);
expect(mocks.consumeSilentCalls).toBe(0);
expect(mocks.persistCalls).toBeGreaterThan(0);
});
it("classifies uploads even when the other editor policy runs on export, not upload", async () => {
// Repro: with a file-producing policy set to export, the auto-run engine dispatches nothing on
// upload - but classification's local pass is independent and must still run on every upload.
securityRunOn.value = "export";
renderHook(() => Harness());
await act(async () => {
await vi.waitFor(
() => {
const classification = latestRuns.filter(
(r) => r.categoryId === "classification" && r.imported,
);
expect(classification).toHaveLength(FILE_COUNT);
},
{ timeout: 8000, interval: 20 },
);
});
// The export policy did not run on upload; nothing versioned in place.
expect(latestRuns.filter((r) => r.categoryId === "security")).toHaveLength(
0,
);
expect(mocks.consumeSilentCalls).toBe(0);
});
});
@@ -1,22 +1,12 @@
import { describe, it, expect, vi, beforeEach, afterEach } from "vitest";
import { renderHook, act } from "@testing-library/react";
// Two active upload policies, so the auto-run should CHAIN them: fire the first on
// the upload, then the second on the first's output. Stub the contexts + network so
// we can drive the dispatch against the REAL run store.
// Controllable AI-engine flag: on by default so classification chains server-side; one
// test flips it off to assert classification is kept OUT of the server chain.
const aiEnabled = vi.hoisted(() => ({ value: true }));
vi.mock("@app/hooks/useAiEngineEnabled", () => ({
useAiEngineEnabled: () => aiEnabled.value,
}));
const fileStubs: {
id: string;
name: string;
derivedFromTool?: boolean;
classificationLabels?: string[];
classificationConfidence?: "none" | "low" | "medium" | "high";
}[] = [];
// Two active file-producing upload policies, so the auto-run should CHAIN them: fire the first on
// the upload, then the second on the first's output. A classification policy is also present to
// assert the engine leaves it alone - annotating policies run themselves (see useClassificationPolicy),
// so they are never in this server chain. Stub the contexts + network to drive dispatch against the
// REAL run store.
const fileStubs: { id: string; name: string; derivedFromTool?: boolean }[] = [];
vi.mock("@app/contexts/FileContext", () => ({
useAllFiles: () => ({ fileStubs }),
useFileManagement: () => ({ addFiles: vi.fn() }),
@@ -27,17 +17,27 @@ vi.mock("@app/hooks/usePolicies", () => ({
policies: {
security: {
configured: true,
runsOnEditor: true,
status: "active",
backendId: "backend-sec",
runOn: "upload",
order: 0,
},
compliance: {
configured: true,
runsOnEditor: true,
status: "active",
backendId: "backend-comp",
runOn: "upload",
order: 1,
},
classification: {
configured: true,
runsOnEditor: true,
status: "active",
backendId: "backend-cls",
runOn: "upload",
order: 1,
order: 2,
},
},
}),
@@ -59,13 +59,14 @@ import { usePolicyAutoRun } from "@app/components/policies/usePolicyAutoRun";
import {
recordRunStart,
updateRun,
getRun,
resetPolicyRuns,
} from "@app/components/policies/policyRunStore";
import { runStoredPolicy, getPolicyRun } from "@app/services/policyApi";
import { fileStorage } from "@app/services/fileStorage";
const runStored = vi.mocked(runStoredPolicy);
const getPolicyRunMock = vi.mocked(getPolicyRun);
const getRunStatus = vi.mocked(getPolicyRun);
const getFile = vi.mocked(fileStorage.getStirlingFile);
/** Reset the shared file list between tests without swapping the array identity. */
@@ -74,12 +75,16 @@ function setFileStubs(next: typeof fileStubs) {
fileStubs.push(...next);
}
/** A completed security run whose imported output is file-1-v2, ready to chain from. */
function seedCompletedSecurityRun() {
function completeRun(
runId: string,
categoryId: string,
fileId: string,
outputFileIds: string[],
) {
recordRunStart({
runId: "run-sec",
categoryId: "security",
fileId: "file-1",
runId,
categoryId,
fileId,
fileName: "doc.pdf",
fileSize: 100,
target: "saas",
@@ -88,11 +93,7 @@ function seedCompletedSecurityRun() {
error: null,
startedAt: 0,
});
updateRun("run-sec", {
status: "COMPLETED",
imported: true,
outputFileIds: ["file-1-v2"],
});
updateRun(runId, { status: "COMPLETED", imported: true, outputFileIds });
}
beforeEach(() => {
@@ -100,7 +101,6 @@ beforeEach(() => {
localStorage.clear();
resetPolicyRuns();
setFileStubs([]);
aiEnabled.value = true;
runStored.mockReset();
getFile.mockReset();
getFile.mockResolvedValue({ size: 100 } as never);
@@ -129,24 +129,8 @@ describe("auto-run ordered chaining", () => {
it("chains the next policy onto a completed run's output", async () => {
// A first-policy run that has completed and imported its output as file-1-v2.
recordRunStart({
runId: "run-sec",
categoryId: "security",
fileId: "file-1",
fileName: "doc.pdf",
fileSize: 100,
target: "saas",
status: "PENDING",
outputs: [],
error: null,
startedAt: 0,
});
updateRun("run-sec", {
status: "COMPLETED",
imported: true,
outputFileIds: ["file-1-v2"],
});
runStored.mockResolvedValue("run-cls");
completeRun("run-sec", "security", "file-1", ["file-1-v2"]);
runStored.mockResolvedValue("run-comp");
renderHook(() => usePolicyAutoRun());
await act(async () => {
@@ -155,182 +139,30 @@ describe("auto-run ordered chaining", () => {
// Fires on the first policy's output and reports that output's own id, not the original's.
expect(runStored).toHaveBeenCalledWith(
"backend-cls",
"backend-comp",
[{ size: 100 }],
"file-1-v2",
);
});
it("escalates a chained output that carries no verdict", async () => {
// The output stub is in the workspace shaped as a new_file-mode delivery (or a
// version made before the upload's verdict landed) produces it: tool-derived,
// labels inherited, NO classificationConfidence. No local pass ever runs on a
// derived file, so waiting for a verdict would skip classification forever —
// it must dispatch to the engine instead.
seedCompletedSecurityRun();
setFileStubs([
{
id: "file-1-v2",
name: "doc.pdf",
derivedFromTool: true,
classificationLabels: ["invoice"],
},
]);
runStored.mockResolvedValue("run-cls");
it("never chains an annotating (classification) policy - it runs itself", async () => {
// The last file-producing policy has completed; the engine's chain ends there. Classification
// is not a wire link in the chain, so nothing dispatches its backend here.
completeRun("run-comp", "compliance", "file-1", ["file-1-v2"]);
runStored.mockResolvedValue("run-x");
renderHook(() => usePolicyAutoRun());
await act(async () => {
await vi.advanceTimersByTimeAsync(1);
});
expect(runStored).toHaveBeenCalledWith(
expect(runStored).not.toHaveBeenCalledWith(
"backend-cls",
[{ size: 100 }],
"file-1-v2",
expect.anything(),
expect.anything(),
);
});
it("chains classification onto an output that inherited an unsure verdict", async () => {
// The default (new_version) delivery: createChildStub copies the parent's
// verdict onto the output, so a low confidence rides through and escalates.
seedCompletedSecurityRun();
setFileStubs([
{
id: "file-1-v2",
name: "doc.pdf",
derivedFromTool: true,
classificationLabels: ["invoice"],
classificationConfidence: "low",
},
]);
runStored.mockResolvedValue("run-cls");
renderHook(() => usePolicyAutoRun());
await act(async () => {
await vi.advanceTimersByTimeAsync(1);
});
expect(runStored).toHaveBeenCalledWith(
"backend-cls",
[{ size: 100 }],
"file-1-v2",
);
});
it("lets an inherited confident verdict stand — no engine call for the chained output", async () => {
seedCompletedSecurityRun();
setFileStubs([
{
id: "file-1-v2",
name: "doc.pdf",
derivedFromTool: true,
classificationLabels: ["invoice"],
classificationConfidence: "high",
},
]);
runStored.mockResolvedValue("run-cls");
renderHook(() => usePolicyAutoRun());
await act(async () => {
await vi.advanceTimersByTimeAsync(1);
});
expect(
runStored.mock.calls.some(([backendId]) => backendId === "backend-cls"),
).toBe(false);
});
it("still escalates after the local pass has recorded its own run for the file", async () => {
// The regression that made the whole escalation dead in practice: the local heuristic records
// a run for the SAME (classification, file) pair, and recordRunStart claims the dispatch key.
// The auto-run then reads "already dispatched" and skips the server run forever. A
// browser-local run must not claim that key - it is the first pass, not the policy's run.
seedCompletedSecurityRun();
// The local pass ran on the chained output and recorded its own run for it.
recordRunStart({
runId: "local-classification-file-1-v2-123",
categoryId: "classification",
fileId: "file-1-v2",
fileName: "doc.pdf",
fileSize: 100,
target: "local",
browserLocal: true,
status: "COMPLETED",
outputs: [],
error: null,
startedAt: 0,
});
// Its verdict was unsure, so the AI must still be asked.
setFileStubs([
{
id: "file-1-v2",
name: "doc.pdf",
derivedFromTool: true,
classificationConfidence: "low",
},
]);
runStored.mockResolvedValue("run-cls");
renderHook(() => usePolicyAutoRun());
await act(async () => {
await vi.advanceTimersByTimeAsync(1);
});
expect(runStored).toHaveBeenCalledWith(
"backend-cls",
[{ size: 100 }],
"file-1-v2",
);
});
it("does not poll a browser-local run against the server", async () => {
// There is no server-side run to ask about: polling 404s, and MAX_NOT_FOUND consecutive
// misses would mark a local run that actually succeeded as FAILED.
recordRunStart({
runId: "local-classification-file-9-456",
categoryId: "classification",
fileId: "file-9",
fileName: "doc.pdf",
fileSize: 100,
target: "local",
browserLocal: true,
status: "RUNNING",
outputs: [],
error: null,
startedAt: 0,
});
setFileStubs([]);
renderHook(() => usePolicyAutoRun());
await act(async () => {
await vi.advanceTimersByTimeAsync(3000);
});
expect(getPolicyRunMock).not.toHaveBeenCalled();
});
it("keeps classification out of the server chain when the AI engine is off", async () => {
// AI off: classification runs client-side (useClientSideClassification), so the
// server chain must skip it - only the normal (security) policy dispatches.
aiEnabled.value = false;
setFileStubs([{ id: "file-1", name: "doc.pdf" }]);
runStored.mockResolvedValue("run-sec");
renderHook(() => usePolicyAutoRun());
await act(async () => {
await vi.advanceTimersByTimeAsync(1);
});
expect(runStored).toHaveBeenCalledWith(
"backend-sec",
[{ size: 100 }],
"file-1",
);
expect(
runStored.mock.calls.some(([backendId]) => backendId === "backend-cls"),
).toBe(false);
});
it("never dispatches on a file marked derivedFromTool", async () => {
// A policy run is billed, so this gate is what stops `importOutputs` re-enforcing a policy on
// its own output forever. If this fails, fix the gate rather than the test.
@@ -346,4 +178,53 @@ describe("auto-run ordered chaining", () => {
expect(runStored).not.toHaveBeenCalled();
});
it("never polls a browser-local run (no server behind it) so its success can't 404 to FAILED", async () => {
getRunStatus.mockResolvedValue({
runId: "srv-1",
policyId: null,
status: "COMPLETED",
currentStep: 1,
stepCount: 1,
error: null,
outputs: [],
} as never);
// A browser-local heuristic run and a real server run, both left in flight.
recordRunStart({
runId: "local-1",
categoryId: "classification",
fileId: "f1",
fileName: "d.pdf",
fileSize: 1,
target: "local",
browserLocal: true,
status: "RUNNING",
outputs: [],
error: null,
startedAt: 0,
});
recordRunStart({
runId: "srv-1",
categoryId: "security",
fileId: "f2",
fileName: "d.pdf",
fileSize: 1,
target: "saas",
status: "RUNNING",
outputs: [],
error: null,
startedAt: 0,
});
renderHook(() => usePolicyAutoRun());
await act(async () => {
await vi.advanceTimersByTimeAsync(700); // past the first poll (500ms)
});
const polled = getRunStatus.mock.calls.map((c) => c[0]);
expect(polled).toContain("srv-1"); // the server run is polled…
expect(polled).not.toContain("local-1"); // …the browser-local one never is
// And its success is left intact, not flipped to FAILED by a 404 streak.
expect(getRun("local-1")?.status).toBe("RUNNING");
});
});
@@ -1,155 +0,0 @@
/**
* The default shipped setup: Classification is the ONLY upload policy, so it dispatches directly
* on the upload rather than through the chain. This is the configuration the escalation was built
* for, and the one where it was completely dead: the browser-side first pass records its own run
* for the same (classification, file) pair, and recordRunStart claims the dispatch key, so the
* auto-run read "already dispatched" and never asked the AI - whatever the verdict said.
*
* Driven against the REAL run store; mocking the store is what let the regression through.
*/
import { describe, it, expect, vi, beforeEach, afterEach } from "vitest";
import { renderHook, act } from "@testing-library/react";
vi.mock("@app/hooks/useAiEngineEnabled", () => ({
useAiEngineEnabled: () => true,
}));
const fileStubs: {
id: string;
name: string;
derivedFromTool?: boolean;
classificationLabels?: string[];
classificationConfidence?: "none" | "low" | "medium" | "high";
}[] = [];
vi.mock("@app/contexts/FileContext", () => ({
useAllFiles: () => ({ fileStubs }),
useFileManagement: () => ({ addFiles: vi.fn() }),
useFileContext: () => ({ consumeFiles: vi.fn() }),
}));
vi.mock("@app/hooks/usePolicies", () => ({
usePolicies: () => ({
policies: {
classification: {
configured: true,
status: "active",
backendId: "backend-cls",
runOn: "upload",
order: 0,
},
},
}),
}));
vi.mock("@app/services/policyApi", () => ({
runStoredPolicy: vi.fn(),
getPolicyRun: vi.fn(),
downloadPolicyOutput: vi.fn(),
resolvePolicyRunTarget: () => "saas",
}));
vi.mock("@app/services/fileStorage", () => ({
fileStorage: { getStirlingFile: vi.fn(), getStirlingFileStub: vi.fn() },
}));
vi.mock("@app/contexts/IndexedDBContext", () => ({
useIndexedDB: () => ({ bumpRevision: vi.fn() }),
}));
import { usePolicyAutoRun } from "@app/components/policies/usePolicyAutoRun";
import {
recordRunStart,
resetPolicyRuns,
} from "@app/components/policies/policyRunStore";
import { runStoredPolicy } from "@app/services/policyApi";
import { fileStorage } from "@app/services/fileStorage";
const runStored = vi.mocked(runStoredPolicy);
const getFile = vi.mocked(fileStorage.getStirlingFile);
function setFileStubs(next: typeof fileStubs) {
fileStubs.length = 0;
fileStubs.push(...next);
}
/**
* Exactly what useClientSideClassification does when its heuristic pass finishes: a run row for
* the activity feed, categorised as classification, for the file it just read.
*/
function recordLocalPassFor(fileId: string) {
recordRunStart({
runId: `local-classification-${fileId}-1`,
categoryId: "classification",
fileId,
fileName: "low-confidence-classification-test.pdf",
fileSize: 1460,
target: "local",
browserLocal: true,
status: "COMPLETED",
outputs: [],
error: null,
startedAt: 0,
});
}
beforeEach(() => {
vi.useFakeTimers();
resetPolicyRuns();
runStored.mockReset();
runStored.mockResolvedValue("run-cls");
getFile.mockReset();
getFile.mockResolvedValue({ size: 1460 } as never);
setFileStubs([]);
});
afterEach(() => vi.useRealTimers());
async function render() {
renderHook(() => usePolicyAutoRun());
await act(async () => {
await vi.advanceTimersByTimeAsync(1);
});
}
describe("classification escalation (single-policy setup)", () => {
it("asks the AI about an unsure verdict even though the local pass already ran", async () => {
// low-confidence-classification-test.pdf: the heuristic emits labels but only at "low".
recordLocalPassFor("file-1");
setFileStubs([
{
id: "file-1",
name: "low-confidence-classification-test.pdf",
classificationLabels: ["contract", "invoice"],
classificationConfidence: "low",
},
]);
await render();
expect(runStored).toHaveBeenCalledWith(
"backend-cls",
[{ size: 1460 }],
"file-1",
);
});
it("leaves a confident local verdict alone (no engine call, no charge)", async () => {
recordLocalPassFor("file-2");
setFileStubs([
{
id: "file-2",
name: "invoice.pdf",
classificationLabels: ["invoice"],
classificationConfidence: "high",
},
]);
await render();
expect(runStored).not.toHaveBeenCalled();
});
it("waits for the verdict rather than racing the local pass", async () => {
// No verdict yet on a plain upload: dispatching now would pay for an answer the free
// first pass is about to produce. The effect re-runs when the verdict lands.
setFileStubs([{ id: "file-3", name: "unknown.pdf" }]);
await render();
expect(runStored).not.toHaveBeenCalled();
});
});
@@ -37,6 +37,7 @@ vi.mock("@app/hooks/usePolicies", () => ({
policies: {
security: {
configured: true,
runsOnEditor: true,
status: "active",
backendId: "backend-1",
runOn: "upload",
@@ -36,11 +36,6 @@ const mocks = vi.hoisted(() => ({
bumpRevision: vi.fn(),
}));
// Classification chains server-side only when the AI engine is on (else it runs
// client-side); this race is in the server import path, so force the engine on.
vi.mock("@app/hooks/useAiEngineEnabled", () => ({
useAiEngineEnabled: () => true,
}));
vi.mock("@app/contexts/FileContext", () => ({
useAllFiles: () => ({ fileStubs: mocks.workspace }),
useFileManagement: () => ({
@@ -57,6 +52,7 @@ vi.mock("@app/hooks/usePolicies", () => ({
policies: {
classification: {
configured: true,
runsOnEditor: true,
status: "active",
backendId: "backend-classification",
runOn: "upload",
@@ -93,6 +89,8 @@ import { usePolicyAutoRun } from "@app/components/policies/usePolicyAutoRun";
import {
usePolicyRuns,
resetPolicyRuns,
recordRunStart,
updateRun,
} from "@app/components/policies/policyRunStore";
import type { PolicyRunRecord } from "@app/components/policies/policyRunStore";
@@ -142,6 +140,26 @@ beforeEach(() => {
error: null,
outputs: [{ fileId: "backend-out-0", fileName: "doc.pdf" }],
});
// Classification now dispatches its own AI run (see usePolicyLocalPasses); this suite exercises
// the auto-run engine's generic import/label-stamping path, so seed a completed classification run
// for it to pick up rather than driving a dispatch.
recordRunStart({
runId: "run-0",
categoryId: "classification",
fileId: "file-0",
fileName: "doc.pdf",
fileSize: 100,
target: "saas",
status: "PENDING",
outputs: [],
error: null,
startedAt: 0,
});
updateRun("run-0", {
status: "COMPLETED",
outputs: [{ fileId: "backend-out-0", fileName: "doc.pdf" }],
});
});
async function settleImport(timeout = 8000) {
@@ -31,6 +31,7 @@ vi.mock("@app/hooks/usePolicies", () => ({
policies: {
security: {
configured: true,
runsOnEditor: true,
status: "active",
backendId: "backend-security",
runOn: "upload",
@@ -13,6 +13,7 @@ vi.mock("@app/hooks/usePolicies", () => ({
policies: {
security: {
configured: true,
runsOnEditor: true,
status: "active",
backendId: "backend-1",
runOn: "upload",
@@ -14,11 +14,9 @@ import { refreshNotificationsNow } from "@app/hooks/useNotifications";
import { useIndexedDB } from "@app/contexts/IndexedDBContext";
import i18n from "@app/i18n";
import {
runStoredPolicy,
getPolicyRun,
listPolicyRuns,
downloadPolicyOutput,
resolvePolicyRunTarget,
} from "@app/services/policyApi";
import type {
PolicyRunStatus,
@@ -29,26 +27,19 @@ import type { FileId } from "@app/types/file";
import { createStirlingFilesAndStubs } from "@app/services/fileStubHelpers";
import { readClassificationLabelsFromFile } from "@app/services/fileClassification";
import {
orderedRewritingCategories,
policyDeliversOutputFiles,
policyRequiresAiEngine,
policyRewritesDocument,
shouldDispatchToAi,
} from "@app/data/classificationPolicy";
import {
acquireDispatchSlot,
releaseDispatchSlot,
} from "@app/components/policies/dispatchSemaphore";
import { runPolicyOnFile } from "@app/services/policyDispatch";
import type { StirlingFile, StirlingFileStub } from "@app/types/fileContext";
import type { PoliciesByCategory } from "@app/types/policies";
import { usePolicies } from "@app/hooks/usePolicies";
import { useAiEngineEnabled } from "@app/hooks/useAiEngineEnabled";
import {
addReconciledRun,
dispatchKey,
getRun,
isDispatched,
markDispatched,
recordRunStart,
removeRun,
updateRun,
usePolicyRuns,
@@ -107,11 +98,6 @@ function failRun(runId: string, message: string): void {
updateRun(runId, { status: "FAILED", error: message, errorCode: null });
}
/** Wait for an upload's bytes to land in IndexedDB (~5s): the stub surfaces in the
* file list before its bytes are committed, so an eager fetch would miss the file. */
const FILE_WAIT_TRIES = 20;
const FILE_WAIT_MS = 250;
/** A policy that changed nothing completes with no output; left unimported its badge
* and blocking overlay spin forever. */
export function finishedWithNothingToDeliver(run: PolicyRunRecord): boolean {
@@ -138,7 +124,6 @@ export function usePolicyAutoRun(): void {
const { consumeFiles } = useFileContext();
const { bumpRevision } = useIndexedDB();
const { policies } = usePolicies();
const aiEnabled = useAiEngineEnabled();
const runs = usePolicyRuns();
// Read in the import effect via ref, not as a dependency: delivery mutates fileStubs,
// so depending on them would re-fire the effect on its own delivery (infinite cascade).
@@ -155,33 +140,13 @@ export function usePolicyAutoRun(): void {
// sentinel for a saas listener to open the modal. Deduped per run.
const firedLimitModal = useRef<Set<string>>(new Set());
// Active upload policies in chain order, so effects accumulate instead of racing to fork
// the same version. Mirrors the dispatch filter so the chain honours the same eligibility.
// The file-producing upload policies this engine dispatches and chains, in run order, so effects
// accumulate instead of racing to fork the same version. Annotating policies (classification) are
// absent by design: they run themselves (local pass, then AI escalation), so the engine never sees
// their two ways to run.
const orderedUploadCategories = useMemo(
() =>
Object.entries(policies)
.filter(
([id, s]) =>
s.configured &&
s.status === "active" &&
s.backendId &&
(!s.sources ||
s.sources.length === 0 ||
s.sources.includes("editor")) &&
(s.runOn ?? "upload") === "upload" &&
// An escalation-only policy has nothing to do with no engine to escalate to.
!(policyRequiresAiEngine(id) && !aiEnabled),
)
// Annotating policies run last: a rewriting one after them would fork a new
// version from the pre-annotation state and drop their labels.
.sort(([idA, a], [idB, b]) => {
const ra = policyRewritesDocument(idA) ? 0 : 1;
const rb = policyRewritesDocument(idB) ? 0 : 1;
if (ra !== rb) return ra - rb;
return (a.order ?? 0) - (b.order ?? 0);
})
.map(([id]) => id),
[policies, aiEnabled],
() => orderedRewritingCategories(policies),
[policies],
);
// Chain-continuations handled this session, so the next policy fires once per run.
@@ -190,9 +155,6 @@ export function usePolicyAutoRun(): void {
// Latest policies, read from inside the stable retry callback (which has no deps).
const policiesRef = useRef(policies);
policiesRef.current = policies;
// Latest stubs for the chaining effect, which keys off runs and must not depend on stubs.
const stubsRef = useRef(fileStubs);
stubsRef.current = fileStubs;
// Per-file (dispatchKey) count of consecutive queue-rejection retries, so backoff escalates and
// eventually gives up. Survives the run-id changing on each retry; reset on any real outcome.
const queueRetries = useRef<Map<string, number>>(new Map());
@@ -272,8 +234,6 @@ export function usePolicyAutoRun(): void {
) {
continue;
}
// A confident local verdict stands; only an unsure one is escalated to the engine.
if (!shouldDispatchToAi(firstCategory, stub)) continue;
dispatching.current.add(key);
void runPolicyOnFile(firstCategory, backendId, stub.id, stub.name)
.catch(() => {
@@ -307,13 +267,6 @@ export function usePolicyAutoRun(): void {
// would otherwise silently skip the next policy on outputs 2..N.
for (const outputId of outputIds) {
if (isDispatched(nextCategory, outputId as FileId)) continue;
const outputStub = stubsRef.current.find((s) => s.id === outputId);
// The output's inherited verdict decides here and now (no local pass ever runs
// on a derived file, so there is nothing to defer to): a confident one stands,
// anything else - including no verdict at all, e.g. a new_file-mode delivery -
// escalates. A stub not yet in the snapshot falls through to dispatch too.
if (outputStub && !shouldDispatchToAi(nextCategory, outputStub))
continue;
void runPolicyOnFile(
nextCategory,
backendId,
@@ -323,15 +276,18 @@ export function usePolicyAutoRun(): void {
).catch(() => {});
}
}
}, [runs, policies, orderedUploadCategories, fileStubs]);
}, [runs, policies, orderedUploadCategories]);
// Poll each in-flight run to a terminal state.
// Poll each in-flight run to a terminal state. A browser-local run (the classification heuristic's
// first pass) has no server run behind it, so polling it 404s and would flip its success to FAILED.
useEffect(() => {
for (const run of runs) {
// A browser-local run has no server-side status: polling it 404s (and after MAX_NOT_FOUND
// marks a run that actually succeeded as failed). Its own pass settles it.
if (run.browserLocal) continue;
if (isTerminal(run.status) || polling.current.has(run.runId)) continue;
if (
run.browserLocal ||
isTerminal(run.status) ||
polling.current.has(run.runId)
)
continue;
polling.current.add(run.runId);
void poll(run.runId, onRunFinished).finally(() =>
polling.current.delete(run.runId),
@@ -377,6 +333,7 @@ export function usePolicyAutoRun(): void {
void importOutputs(run, {
addFiles,
consumeFiles,
policyName: policies[run.categoryId]?.name,
updateStirlingFileStub,
bumpRevision,
outputMode,
@@ -427,6 +384,8 @@ interface ImportContext {
bumpRevision: () => void;
/** "new_file" adds the output as a separate file; "new_version" versions the input. */
outputMode: "new_file" | "new_version";
/** The policy's name, shown in version history instead of the generic "automate" tool. */
policyName?: string;
/** Rename rule. Empty → keep the input's filename. */
outputName: string;
/** Rename position around the base filename; defaults to "suffix" when absent. */
@@ -744,6 +703,7 @@ async function importOutputs(
files,
parentStub,
"automate",
ctx.policyName,
);
// Transitive provenance for the PERSISTED record, mirroring what the
// CONSUME_FILES reducer computes for workspace state: the output derives
@@ -773,7 +733,7 @@ async function importOutputs(
// Mark the outputs handled BEFORE adding them (belt-and-suspenders session
// guard on top of derivedFromTool) so the auto-run never enforces the policy
// on its own output — that would version endlessly in a loop.
for (const s of categorized) markHandled(s.id);
for (const s of categorized) markHandled(s.id as string);
deliveredIds = categorized.map((s) => s.id as string);
if (ctx.parentStub) {
// Input is in the active workspace: version it in place, silently — the
@@ -804,7 +764,7 @@ async function importOutputs(
derivedFromTool: true,
});
// Belt-and-suspenders session guard on top of derivedFromTool.
for (const f of added) markHandled(f.fileId);
for (const f of added) markHandled(f.fileId as string);
deliveredIds = added.map((f) => f.fileId as string);
// Mark each new-file output as tool-derived (the versioned path gets this from the
// CONSUME_FILES reducer; the addFiles path doesn't). This is the real loop guard: the dispatch
@@ -850,75 +810,6 @@ async function importOutputs(
}
}
/** Resolve the file's bytes, fire a backend run, and record it. */
async function runPolicyOnFile(
categoryId: string,
backendId: string,
fileId: FileId,
fileName: string,
// Chained (downstream) dispatch — jumps the dispatch queue so a file mid-chain
// finishes its flow before new files start (see acquireDispatchSlot).
priority = false,
): Promise<void> {
// A freshly-uploaded file's bytes are written to IndexedDB asynchronously, so
// its stub can appear in the file list a beat before getStirlingFile resolves
// it. Wait briefly rather than bail — and DON'T mark dispatched until we hold
// the file, or a too-early miss would skip enforcement on that file forever.
// (The caller's in-flight guard prevents double-dispatch during this wait.)
// A transient IndexedDB error is treated as a miss (not a throw), so it retries
// and then marks dispatched rather than rejecting into a hot re-dispatch loop.
const tryGetFile = async (): Promise<StirlingFile | null> => {
try {
return await fileStorage.getStirlingFile(fileId);
} catch {
return null;
}
};
let file = await tryGetFile();
for (let i = 0; i < FILE_WAIT_TRIES && !file; i++) {
await delay(FILE_WAIT_MS);
file = await tryGetFile();
}
if (!file) {
// File genuinely gone (removed before it could run) — mark so we don't loop.
markDispatched(categoryId, fileId);
return;
}
// Bounded upload window — see MAX_CONCURRENT_DISPATCHES. Only the POST is
// gated; the IDB wait above never holds a slot.
await acquireDispatchSlot(priority);
try {
const target = resolvePolicyRunTarget();
// Recorded against a document this browser can resolve. One file per run, which is the only
// shape the server keeps a reference for.
const runId = await runStoredPolicy(backendId, [file], fileId);
// recordRunStart marks this (policy, file) dispatched as it records the run.
recordRunStart({
runId,
categoryId,
fileId,
fileName,
fileSize: file.size,
target,
status: "PENDING",
outputs: [],
error: null,
startedAt: Date.now(),
});
} catch (err) {
// Dispatch failed (e.g. policy deleted/404 or backend offline). Mark dispatched so we don't hammer;
// the absent run simply won't appear in the activity feed. If the backend did
// start a run we never recorded, reconcileServerRuns rediscovers it.
console.debug(
`[PolicyAutoRun] Failed to dispatch policy ${categoryId} (${backendId}):`,
err,
);
markDispatched(categoryId, fileId);
} finally {
releaseDispatchSlot();
}
}
/**
* Poll a run's status until it reaches a terminal state (or the budget). Calls {@code onTerminal} once
* with the final view when it terminates the caller uses that to pop the usage-limit modal when a
@@ -1,5 +1,6 @@
// Delivery guarantees of the client-side classification hook, driving the real
// policyRunStore and mocking only IO (storage, the heuristic engine, the meter).
// The Classification policy hook: classifies each settled document locally and escalates an unsure
// verdict to the AI engine itself. Drives the real policyRunStore and mocks only IO (storage, the
// heuristic engine, the meter, the shared dispatch primitive).
import { describe, it, expect, vi, beforeEach } from "vitest";
import { renderHook, waitFor } from "@testing-library/react";
@@ -16,6 +17,9 @@ interface TestStub {
classificationLabels?: string[];
}
const aiEnabled = vi.hoisted(() => ({ value: false }));
const runOn = vi.hoisted(() => ({ value: "upload" as "upload" | "export" }));
const mocks = vi.hoisted(() => ({
workspace: [] as Array<{
id: string;
@@ -31,6 +35,7 @@ const mocks = vi.hoisted(() => ({
updateFileMetadata: vi.fn(async (_id: string, _updates: unknown) => true),
classify: vi.fn(),
meter: vi.fn(),
runPolicyOnFile: vi.fn(),
}));
vi.mock("@app/contexts/AppConfigContext", () => ({
@@ -40,15 +45,20 @@ vi.mock("@app/hooks/useClassificationEnabled", () => ({
useClassificationEnabled: () => true,
}));
vi.mock("@app/hooks/useAiEngineEnabled", () => ({
useAiEngineEnabled: () => false,
useAiEngineEnabled: () => aiEnabled.value,
}));
vi.mock("@app/services/policyDispatch", () => ({
runPolicyOnFile: (...args: unknown[]) => mocks.runPolicyOnFile(...args),
}));
vi.mock("@app/hooks/usePolicies", () => ({
usePolicies: () => ({
policies: {
classification: {
configured: true,
runsOnEditor: true,
status: "active",
backendId: "backend-classification",
runOn: runOn.value,
sources: ["editor"],
},
},
@@ -77,10 +87,7 @@ vi.mock("@app/services/classificationMeter", () => ({
meterClassificationRun: (payload: unknown) => mocks.meter(payload),
}));
import {
useClientSideClassification,
LOCAL_METER_CATEGORY,
} from "@app/components/policies/useClientSideClassification";
import { usePolicyLocalPasses } from "@app/components/policies/usePolicyLocalPasses";
// Run idle callbacks immediately so batches start without timer waits.
vi.stubGlobal("requestIdleCallback", (cb: () => void) => {
@@ -98,7 +105,7 @@ const stub = (id: string, extra: Partial<TestStub> = {}): TestStub => ({
const fakeFile = (id: string) => new File([id], `${id}.pdf`);
describe("useClientSideClassification delivery", () => {
describe("usePolicyLocalPasses delivery", () => {
beforeEach(() => {
localStorage.clear();
resetPolicyRuns();
@@ -113,6 +120,10 @@ describe("useClientSideClassification delivery", () => {
fakeFile(id),
);
mocks.classify.mockReset();
aiEnabled.value = false;
runOn.value = "upload";
mocks.runPolicyOnFile.mockReset();
mocks.runPolicyOnFile.mockResolvedValue(undefined);
});
it("classifies pending uploads, writes labels, and meters once per file", async () => {
@@ -121,7 +132,7 @@ describe("useClientSideClassification delivery", () => {
labels: [file.name.startsWith("a") ? "invoice" : "resume"],
}));
renderHook(() => useClientSideClassification());
renderHook(() => usePolicyLocalPasses());
await waitFor(() =>
expect(mocks.updateStirlingFileStub).toHaveBeenCalledTimes(2),
@@ -144,7 +155,7 @@ describe("useClientSideClassification delivery", () => {
);
mocks.workspace = [stub("a")];
const { rerender } = renderHook(() => useClientSideClassification());
const { rerender } = renderHook(() => usePolicyLocalPasses());
await waitFor(() => expect(mocks.classify).toHaveBeenCalledTimes(1));
// A new upload mid-classify re-fires the effect and cancels the in-flight
@@ -172,7 +183,7 @@ describe("useClientSideClassification delivery", () => {
mocks.workspace = [stub("plain")];
mocks.classify.mockResolvedValue({ labels: [] });
renderHook(() => useClientSideClassification());
renderHook(() => usePolicyLocalPasses());
await waitFor(() =>
expect(mocks.updateStirlingFileStub).toHaveBeenCalledWith("plain", {
@@ -184,13 +195,12 @@ describe("useClientSideClassification delivery", () => {
});
it("heals a previously-dispatched file whose result was lost, without re-metering", async () => {
// A past session classified + metered this file but the delivery was lost. The marker is the
// local-meter key, NOT the classification dispatch key - that one belongs to the server run.
markDispatched(LOCAL_METER_CATEGORY, "lost");
// A past session classified + metered this file but the delivery was lost.
markDispatched("classification", "lost");
mocks.workspace = [stub("lost")];
mocks.classify.mockResolvedValue({ labels: ["bank-statement"] });
renderHook(() => useClientSideClassification());
renderHook(() => usePolicyLocalPasses());
await waitFor(() =>
expect(mocks.updateStirlingFileStub).toHaveBeenCalledWith("lost", {
@@ -207,7 +217,7 @@ describe("useClientSideClassification delivery", () => {
mocks.workspace = [stub("corrupt")];
mocks.classify.mockRejectedValue(new Error("bad pdf"));
renderHook(() => useClientSideClassification());
renderHook(() => usePolicyLocalPasses());
await waitFor(() =>
expect(warn).toHaveBeenCalledWith(
@@ -229,7 +239,7 @@ describe("useClientSideClassification delivery", () => {
mocks.workspace = [stub("early")];
mocks.classify.mockResolvedValue({ labels: ["invoice"] });
const { rerender } = renderHook(() => useClientSideClassification());
const { rerender } = renderHook(() => usePolicyLocalPasses());
await new Promise((r) => setTimeout(r, 50));
expect(mocks.classify).not.toHaveBeenCalled();
@@ -250,10 +260,94 @@ describe("useClientSideClassification delivery", () => {
stub("verdict", { classificationLabels: [] }),
];
renderHook(() => useClientSideClassification());
renderHook(() => usePolicyLocalPasses());
// Nothing to classify; give the (immediate) idle path a beat to prove it.
await new Promise((r) => setTimeout(r, 50));
expect(mocks.classify).not.toHaveBeenCalled();
});
it("escalates an unsure local verdict to the AI engine itself", async () => {
aiEnabled.value = true;
mocks.workspace = [stub("a")];
mocks.classify.mockResolvedValue({
labels: ["invoice"],
confidence: "low",
});
renderHook(() => usePolicyLocalPasses());
await waitFor(() =>
expect(mocks.runPolicyOnFile).toHaveBeenCalledWith(
"classification",
"backend-classification",
"a",
"a.pdf",
),
);
// The local verdict is still delivered before escalation.
expect(mocks.updateStirlingFileStub).toHaveBeenCalledWith("a", {
classificationLabels: ["invoice"],
classificationConfidence: "low",
});
});
it("lets a confident local verdict stand without asking the AI", async () => {
aiEnabled.value = true;
mocks.workspace = [stub("a")];
mocks.classify.mockResolvedValue({
labels: ["invoice"],
confidence: "high",
});
renderHook(() => usePolicyLocalPasses());
await waitFor(() =>
expect(mocks.updateStirlingFileStub).toHaveBeenCalledWith("a", {
classificationLabels: ["invoice"],
classificationConfidence: "high",
}),
);
expect(mocks.runPolicyOnFile).not.toHaveBeenCalled();
});
it("never escalates when the AI engine is off, however unsure the verdict", async () => {
aiEnabled.value = false;
mocks.workspace = [stub("a")];
mocks.classify.mockResolvedValue({
labels: ["invoice"],
confidence: "none",
});
renderHook(() => usePolicyLocalPasses());
await waitFor(() =>
expect(mocks.updateStirlingFileStub).toHaveBeenCalledWith("a", {
classificationLabels: ["invoice"],
classificationConfidence: "none",
}),
);
expect(mocks.runPolicyOnFile).not.toHaveBeenCalled();
});
it("does not run on upload when the policy is set to run on export", async () => {
// An export-time policy is enforced by the export path, not this upload engine, so an upload
// must not classify, meter, or escalate.
runOn.value = "export";
aiEnabled.value = true;
mocks.workspace = [stub("a")];
mocks.classify.mockResolvedValue({
labels: ["invoice"],
confidence: "low",
});
renderHook(() => usePolicyLocalPasses());
// Give the (immediate) idle path a beat to prove it stays silent.
await new Promise((r) => setTimeout(r, 50));
expect(mocks.classify).not.toHaveBeenCalled();
expect(mocks.updateStirlingFileStub).not.toHaveBeenCalled();
expect(mocks.meter).not.toHaveBeenCalled();
expect(mocks.runPolicyOnFile).not.toHaveBeenCalled();
});
});
@@ -0,0 +1,144 @@
/**
* Generic engine for policies' browser-side fast paths. Any active editor policy that declares a
* {@link LocalPass} has it run here: eligible files are classified/processed locally, the returned
* fields are written to the stub, and the policy's server run is dispatched only if the pass says it
* is still needed (and the AI engine, if the policy needs it, is on).
*/
import { useEffect, useMemo, useRef, useState } from "react";
import { useAllFiles, useFileManagement } from "@app/contexts/FileContext";
import { useAppConfig } from "@app/contexts/AppConfigContext";
import { useIndexedDB } from "@app/contexts/IndexedDBContext";
import { fileStorage } from "@app/services/fileStorage";
import { useAiEngineEnabled } from "@app/hooks/useAiEngineEnabled";
import { scheduleIdle } from "@app/utils/scheduleIdle";
import { usePolicies } from "@app/hooks/usePolicies";
import { runPolicyOnFile } from "@app/services/policyDispatch";
import {
localPassFor,
type LocalPass,
} from "@app/components/policies/policyLocalPass";
import { policyRequiresAiEngine } from "@app/data/classificationPolicy";
import type { StirlingFileStub } from "@app/types/fileContext";
/** Files processed per idle pass, so a large upload drains over several ticks instead of janking. */
const LOCAL_PASS_BATCH = 3;
interface ActivePass {
categoryId: string;
backendId: string;
pass: LocalPass;
/** When true, the server run is skipped while the AI engine is off (nothing to escalate to). */
requiresAiEngine: boolean;
}
export function usePolicyLocalPasses(): void {
const { fileStubs } = useAllFiles();
const { updateStirlingFileStub } = useFileManagement();
const { bumpRevision } = useIndexedDB();
const { policies } = usePolicies();
const aiEnabled = useAiEngineEnabled();
// Waited on so a verdict is not written and escalated before it is known whether the AI engine
// (which the server run may need) is even available.
const { loading: configLoading } = useAppConfig();
// Files claimed this session, keyed policy+id+lastModified so a new version is retried once. Claimed
// synchronously right before running, so overlapping batches never double-process.
const claimed = useRef<Set<string>>(new Set());
// Bumped after each batch to drain the next one.
const [tick, setTick] = useState(0);
// Active editor upload policies that declare a local fast path.
const passes = useMemo<ActivePass[]>(() => {
const out: ActivePass[] = [];
for (const [categoryId, s] of Object.entries(policies)) {
const active =
s.configured &&
s.status === "active" &&
s.backendId &&
s.runsOnEditor &&
(s.runOn ?? "upload") === "upload";
if (!active) continue;
const pass = localPassFor(categoryId);
if (!pass) continue;
out.push({
categoryId,
backendId: s.backendId as string,
pass,
requiresAiEngine: policyRequiresAiEngine(categoryId),
});
}
return out;
}, [policies]);
useEffect(() => {
if (configLoading || passes.length === 0) return;
const claimKey = (categoryId: string, s: StirlingFileStub) =>
`${categoryId}:${s.id as string}:${s.lastModified ?? 0}`;
// Collect one idle batch of pending (pass, file) work across all passes.
const batch: { active: ActivePass; stub: StirlingFileStub }[] = [];
outer: for (const active of passes) {
for (const stub of fileStubs) {
if (batch.length >= LOCAL_PASS_BATCH) break outer;
if (!active.pass.eligible(stub)) continue;
if (claimed.current.has(claimKey(active.categoryId, stub))) continue;
batch.push({ active, stub });
}
}
if (batch.length === 0) return;
let cancelled = false;
const cancelIdle = scheduleIdle(() => {
// Superseded before starting: the newer effect instance owns the queue.
if (cancelled) return;
void (async () => {
let wrote = false;
for (const { active, stub } of batch) {
const key = claimKey(active.categoryId, stub);
// Re-validate at execution time - another batch may have claimed it since.
if (claimed.current.has(key)) continue;
claimed.current.add(key);
const result = await active.pass.run(stub.id, stub);
// Could not run (e.g. bytes not in storage yet): leave unprocessed so a reload retries.
if (result == null) continue;
// Deliver unconditionally - a re-render must never discard a computed result. Writes are
// idempotent. The engine applies the fields the pass returned without reading them.
updateStirlingFileStub(stub.id, result.stubUpdates);
const ok = await fileStorage.updateFileMetadata(
stub.id,
result.stubUpdates,
);
if (ok) wrote = true;
// Dispatch the server run only if the pass still wants it, and skip it while an
// AI-engine-dependent policy has no engine to reach.
if (
result.needsServerRun &&
!(active.requiresAiEngine && !aiEnabled)
) {
void runPolicyOnFile(
active.categoryId,
active.backendId,
stub.id,
stub.name,
).catch(() => {
// Backstop: runPolicyOnFile handles its own failures.
});
}
}
if (wrote) bumpRevision();
// Drain the next batch; the terminal pass finds nothing pending and stops.
setTick((n) => n + 1);
})();
});
return () => {
cancelled = true;
cancelIdle();
};
}, [
fileStubs,
passes,
aiEnabled,
configLoading,
updateStirlingFileStub,
bumpRevision,
tick,
]);
}
@@ -1,26 +1,23 @@
import { describe, it, expect } from "vitest";
import {
isClassificationCategory,
localVerdictNeedsEscalation,
orderRewritesFirst,
orderedRewritingCategories,
policyDeliversOutputFiles,
policyRequiresAiEngine,
policyRewritesDocument,
shouldDispatchToAi,
} from "@app/data/classificationPolicy";
import type { StirlingFileStub } from "@app/types/fileContext";
import type { PoliciesByCategory } from "@app/types/policies";
const stub = (
confidence?: StirlingFileStub["classificationConfidence"],
): StirlingFileStub =>
({ classificationConfidence: confidence }) as StirlingFileStub;
const derivedStub = (
confidence?: StirlingFileStub["classificationConfidence"],
): StirlingFileStub =>
const rewriter = (order: number) =>
({
derivedFromTool: true,
classificationConfidence: confidence,
}) as StirlingFileStub;
configured: true,
status: "active",
backendId: `backend-${order}`,
runsOnEditor: true,
runOn: "upload",
order,
}) as unknown as PoliciesByCategory[string];
describe("isClassificationCategory", () => {
it("recognises the classification category and nothing else", () => {
@@ -42,11 +39,6 @@ describe("policy capabilities", () => {
expect(policyDeliversOutputFiles("security")).toBe(true);
expect(policyDeliversOutputFiles("classification")).toBe(false);
});
it("marks classification as the AI-escalation policy", () => {
expect(policyRequiresAiEngine("classification")).toBe(true);
expect(policyRequiresAiEngine("security")).toBe(false);
});
});
describe("orderRewritesFirst", () => {
@@ -75,40 +67,52 @@ describe("orderRewritesFirst", () => {
});
});
describe("shouldDispatchToAi", () => {
it("always dispatches a policy that is not classification", () => {
expect(shouldDispatchToAi("security", stub())).toBe(true);
expect(shouldDispatchToAi("security", stub("high"))).toBe(true);
describe("orderedRewritingCategories", () => {
it("lists only file-producing policies, ordered by order, excluding classification", () => {
const policies = {
classification: rewriter(0), // annotating: excluded despite being active
security: rewriter(2),
compliance: rewriter(1),
} as unknown as PoliciesByCategory;
// classification is filtered by policyDeliversOutputFiles, not by the shape above.
expect(orderedRewritingCategories(policies)).toEqual([
"compliance",
"security",
]);
});
it("holds back until the local heuristic has reported", () => {
// Not a skip: dispatching now races the local pass and pays for a free answer;
// the caller re-evaluates once the verdict lands.
expect(shouldDispatchToAi("classification", stub())).toBe(false);
it("excludes inactive, non-editor, export-triggered, and unconfigured policies", () => {
const mixed = {
security: rewriter(0),
inactive: { ...rewriter(1), status: "paused" },
notEditor: { ...rewriter(2), runsOnEditor: false },
onExport: { ...rewriter(3), runOn: "export" },
unconfigured: { ...rewriter(4), configured: false },
noBackend: { ...rewriter(5), backendId: undefined },
} as unknown as PoliciesByCategory;
expect(orderedRewritingCategories(mixed)).toEqual(["security"]);
});
it("is empty when classification is the only policy", () => {
const only = {
classification: rewriter(0),
} as unknown as PoliciesByCategory;
expect(orderedRewritingCategories(only)).toEqual([]);
});
});
describe("localVerdictNeedsEscalation", () => {
it("lets a confident local verdict stand", () => {
expect(shouldDispatchToAi("classification", stub("high"))).toBe(false);
expect(localVerdictNeedsEscalation("high")).toBe(false);
});
it("does not escalate when no verdict has been recorded yet", () => {
expect(localVerdictNeedsEscalation(undefined)).toBe(false);
});
it("escalates anything less than confident", () => {
expect(shouldDispatchToAi("classification", stub("medium"))).toBe(true);
expect(shouldDispatchToAi("classification", stub("low"))).toBe(true);
expect(shouldDispatchToAi("classification", stub("none"))).toBe(true);
});
it("escalates a tool-derived file with no verdict at all", () => {
// A derived file gets no local pass (useClientSideClassification skips it), so
// there is no verdict to wait for: holding back would skip it forever. This is
// the chained case for a new_file-mode output, or a version made before the
// upload's verdict landed.
expect(shouldDispatchToAi("classification", derivedStub())).toBe(true);
});
it("lets a derived file's inherited verdict decide like an upload's own", () => {
expect(shouldDispatchToAi("classification", derivedStub("high"))).toBe(
false,
);
expect(shouldDispatchToAi("classification", derivedStub("low"))).toBe(true);
expect(localVerdictNeedsEscalation("medium")).toBe(true);
expect(localVerdictNeedsEscalation("low")).toBe(true);
expect(localVerdictNeedsEscalation("none")).toBe(true);
});
});
@@ -1,7 +1,9 @@
/**
* Everything specific to the built-in Classification policy, in one module. The generic policy
* runner asks the capability questions below instead of naming classification itself, so a second
* annotating policy needs a change here rather than in the runner.
* Everything specific to the built-in Classification policy, in one module. The generic policy runner
* dispatches and chains only file-producing policies (see {@link orderedRewritingCategories}), and the
* generic local-pass engine runs whatever browser-side fast path a policy declares. Classification's
* fast path (its heuristic) lives in classificationLocalPass; the capability answers below let the
* generic engines treat it without naming it. A second annotating policy is a change here, not there.
*
* These are still keyed on the category id rather than a property each policy declares. That is
* deliberate for now: policies are becoming pipelines with labels behind a separate enforcement
@@ -11,10 +13,8 @@
* mode, and a run result that can carry findings as well as files), not in a flag added here first.
*/
import type {
ClassificationConfidence,
StirlingFileStub,
} from "@app/types/fileContext";
import type { ClassificationConfidence } from "@app/types/fileContext";
import type { PoliciesByCategory } from "@app/types/policies";
/** Catalogue category id of the built-in Classification policy. */
export const CLASSIFICATION_CATEGORY_ID = "classification";
@@ -36,7 +36,10 @@ export function policyDeliversOutputFiles(categoryId: string): boolean {
return policyRewritesDocument(categoryId);
}
/** Whether the policy's server-side run exists only to escalate to the AI engine. */
/**
* Whether the policy's server run needs the AI engine. The local-pass engine skips dispatching such
* a run when the engine is off - there is nothing to escalate to, and the local verdict stands.
*/
export function policyRequiresAiEngine(categoryId: string): boolean {
return isClassificationCategory(categoryId);
}
@@ -49,6 +52,29 @@ export function orderRewritesFirst(categoryIds: string[]): string[] {
];
}
/**
* The active editor upload policies the generic runner dispatches and chains, in run order. Only
* file-producing policies: an annotating policy (classification) has no output to chain onto and
* runs itself, so it is intentionally absent here. Both the runner and the classification policy
* read this - the runner to sequence the chain, classification to know when that chain is done.
*/
export function orderedRewritingCategories(
policies: PoliciesByCategory,
): string[] {
return Object.entries(policies)
.filter(
([id, s]) =>
s.configured &&
s.status === "active" &&
Boolean(s.backendId) &&
s.runsOnEditor &&
(s.runOn ?? "upload") === "upload" &&
policyDeliversOutputFiles(id),
)
.sort(([, a], [, b]) => (a.order ?? 0) - (b.order ?? 0))
.map(([id]) => id);
}
/**
* The one heuristic verdict trusted to stand on its own; anything less escalates to the AI, which
* overwrites it. Deliberately strict - a wrong label costs more than an engine call.
@@ -56,18 +82,12 @@ export function orderRewritesFirst(categoryIds: string[]): string[] {
const TRUSTED_CONFIDENCE: ClassificationConfidence = "high";
/**
* Whether the AI classifier should be asked about this file. For an upload, only once the
* heuristic has reported: dispatching before then races the first pass and bills for an answer it
* was about to produce. A tool-derived file gets no local pass (useClientSideClassification skips
* it) and only ever carries an inherited verdict, so an absent verdict there is permanent -
* escalate rather than wait for a report that will never come.
* Whether a local classification verdict must be escalated to the AI engine. A confident verdict
* stands on its own; anything less is escalated and overwritten. Owned here, alongside the local
* pass that produces the verdict - the runner is not involved.
*/
export function shouldDispatchToAi(
categoryId: string,
stub: StirlingFileStub,
export function localVerdictNeedsEscalation(
confidence: ClassificationConfidence | undefined,
): boolean {
if (!isClassificationCategory(categoryId)) return true;
const confidence = stub.classificationConfidence;
if (confidence == null) return Boolean(stub.derivedFromTool);
return confidence !== TRUSTED_CONFIDENCE;
return confidence != null && confidence !== TRUSTED_CONFIDENCE;
}
@@ -48,7 +48,8 @@ const wizardResult = {
updatedAt: "",
},
fieldValues: {},
sources: ["editor"],
sources: [],
runsOnEditor: true,
scopeTypes: [],
reviewerEmail: "reviewer@x.com",
folder: {
@@ -153,4 +154,76 @@ describe("usePolicies", () => {
});
expect(result.current.policies.ingestion.folderId).toBeTruthy();
});
// A builder pipeline has no category tile, so the reconcile must key it by id to reach the map
// the auto-run iterates.
it("reconciles a builder pipeline that has no category", async () => {
api.store.set("be-pipeline", {
id: "be-pipeline",
name: "My pipeline",
enabled: true,
inputs: [],
steps: [{ operation: "/api/v1/misc/compress-pdf", parameters: {} }],
output: { type: "inline", options: {} },
outputIds: [],
editor: { allowed: true, runOn: "upload" },
} as unknown as { id: string });
const { result } = renderHook(() => usePolicies());
await waitFor(() =>
expect(result.current.policies["be-pipeline"]?.configured).toBe(true),
);
const pipeline = result.current.policies["be-pipeline"];
expect(pipeline.runsOnEditor).toBe(true);
// Not a catalogue tile, so it is deletable rather than a built-in default.
expect(pipeline.isDefault).toBe(false);
});
it("does not put a builder pipeline on the editor unless it opts in", async () => {
api.store.set("be-s3", {
id: "be-s3",
name: "S3 sweep",
enabled: true,
inputs: [],
steps: [{ operation: "/api/v1/misc/compress-pdf", parameters: {} }],
output: { type: "inline", options: {} },
outputIds: [],
} as unknown as { id: string });
const { result } = renderHook(() => usePolicies());
await waitFor(() =>
expect(result.current.policies["be-s3"]?.configured).toBe(true),
);
expect(result.current.policies["be-s3"].runsOnEditor).toBe(false);
});
// Deleting a pipeline on the Pipelines page leaves its cached entry behind. It still satisfies
// every auto-run condition but its backendId is dead, so the dispatch fails, the run never
// completes, and every policy behind it in the chain is skipped on every upload.
it("forgets a builder pipeline the backend no longer has", async () => {
localStorage.setItem(
"stirling-policies-state",
JSON.stringify({
"be-deleted": {
configured: true,
status: "active",
backendId: "be-deleted",
sources: ["editor"],
runsOnEditor: true,
runOn: "upload",
isDefault: false,
},
}),
);
const { result } = renderHook(() => usePolicies());
await waitFor(() =>
expect(result.current.policies["be-deleted"]).toBeUndefined(),
);
// A catalogue tile is never forgotten: it reseeds from the catalogue.
expect(result.current.policies.security).toBeDefined();
});
});
@@ -14,6 +14,7 @@ import {
onPoliciesChange,
updatePolicy,
resetPolicy,
forgetPolicies,
reorderPolicies as persistPolicyOrder,
} from "@app/services/policyStorage";
import { loadPolicyCatalog } from "@app/services/policyCatalog";
@@ -36,7 +37,7 @@ import {
} from "@app/services/policyBackend";
import { reorderPolicies as reorderBackendPolicies } from "@app/services/policyApi";
import { orderRewritesFirst } from "@app/data/classificationPolicy";
import type { PolicyToStore } from "@app/services/policyPipeline";
import { type PolicyToStore } from "@app/services/policyPipeline";
import type {
PoliciesByCategory,
PolicyConfigResult,
@@ -65,6 +66,7 @@ function toStoreRequest(
automation: result.automation,
pipelineSteps: result.pipelineSteps,
sources: result.sources,
runsOnEditor: result.runsOnEditor,
scopeTypes: result.scopeTypes,
reviewerEmail: result.reviewerEmail,
fieldValues: result.fieldValues,
@@ -114,6 +116,20 @@ export function usePolicies() {
backendId: undefined,
};
}
// Builder-made pipelines have no category, so the built-in loop above skips them. They are
// still policies: one set to run on the editor has to reach the auto-run.
for (const [key, decoded] of byCategory) {
if (reconciled[key]) continue;
reconciled[key] = decodedToState(decoded, local[key]?.folderId);
}
// A builder pipeline the backend no longer has was deleted on the Pipelines page. Its cached
// entry keeps a dead backendId that still satisfies the auto-run filter, so the dispatch
// fails, the run never completes, and the chain behind it never advances.
forgetPolicies(
Object.keys(local).filter(
(id) => !reconciled[id] && !byCategory.has(id),
),
);
for (const [id, state] of Object.entries(reconciled)) {
updatePolicy(id, state);
}
@@ -157,6 +173,7 @@ export function usePolicies() {
backendId,
fieldValues: result.fieldValues,
sources: result.sources,
runsOnEditor: result.runsOnEditor,
scopeTypes: result.scopeTypes,
reviewerEmail: result.reviewerEmail,
outputMode: result.folder.outputMode,
@@ -194,6 +211,7 @@ export function usePolicies() {
backendId,
fieldValues: result.fieldValues,
sources: result.sources,
runsOnEditor: result.runsOnEditor,
scopeTypes: result.scopeTypes,
reviewerEmail: result.reviewerEmail,
outputMode: result.folder.outputMode,
@@ -246,6 +264,7 @@ export function usePolicies() {
},
pipelineSteps: result.pipelineSteps,
sources: result.sources,
runsOnEditor: result.runsOnEditor,
scopeTypes: result.scopeTypes,
reviewerEmail: result.reviewerEmail,
fieldValues: result.fieldValues,
@@ -259,6 +278,7 @@ export function usePolicies() {
backendId,
fieldValues: result.fieldValues,
sources: result.sources,
runsOnEditor: result.runsOnEditor,
scopeTypes: result.scopeTypes,
reviewerEmail: result.reviewerEmail,
outputMode: result.folder.outputMode,
@@ -8,6 +8,7 @@ const FULL_STATE: PolicyDecodedState = {
enabled: true,
categoryId: "security",
sources: ["editor", "gdrive"],
runsOnEditor: true,
scopeTypes: ["Contracts", "Invoices"],
reviewerEmail: "admin@example.com",
fieldValues: { auditTrail: true, frameworks: ["HIPAA"] },
@@ -26,10 +27,18 @@ const FULL_STATE: PolicyDecodedState = {
};
describe("toWirePolicy", () => {
it("sets trigger to null", () => {
it("sets trigger to null for categories the editor fires itself", () => {
expect(toWirePolicy(FULL_STATE).trigger).toBeNull();
});
it("marks the sharing category by categoryId, not a trigger", () => {
// Egress has no source to hang a trigger on, so the backend finds the
// policy by the category it was authored under.
const wire = toWirePolicy({ ...FULL_STATE, categoryId: "sharing" });
expect(wire.trigger).toBeNull();
expect(wire.output.options.categoryId).toBe("sharing");
});
it("sets output.type to inline", () => {
expect(toWirePolicy(FULL_STATE).output.type).toBe("inline");
});
@@ -44,6 +53,26 @@ describe("toWirePolicy", () => {
expect(opts.position).toBe("prefix");
});
it("sends the editor block so a save never drops editor participation", () => {
expect(toWirePolicy(FULL_STATE).editor).toEqual({
allowed: true,
runOn: "upload",
});
expect(toWirePolicy({ ...FULL_STATE, runsOnEditor: false }).editor).toEqual(
{ allowed: false, runOn: "upload" },
);
});
it("keeps editor participation that empty sources would have re-derived away", () => {
// The seeded Classification policy: editor-run, no sources.
const wire = toWirePolicy({
...FULL_STATE,
sources: [],
runsOnEditor: true,
});
expect(wire.editor?.allowed).toBe(true);
});
it("preserves steps at the top level", () => {
const wire = toWirePolicy(FULL_STATE);
expect(wire.steps).toEqual(FULL_STATE.steps);
@@ -69,15 +98,24 @@ describe("fromWirePolicy → round-trip", () => {
expect(decoded.steps).toEqual(FULL_STATE.steps);
});
it("defaults a missing runOn to the category default (security → export)", () => {
const wire = toWirePolicy(FULL_STATE);
// The moment has two possible homes now (the `editor` block, and the legacy
// options bag), so "nothing stored" means clearing both.
const withNoStoredRunOn = (state: PolicyDecodedState) => {
const wire = toWirePolicy(state);
delete (wire.output.options as Record<string, unknown>).runOn;
expect(fromWirePolicy(wire).runOn).toBe("export");
delete wire.editor;
return wire;
};
it("defaults a missing runOn to the category default (security → export)", () => {
expect(fromWirePolicy(withNoStoredRunOn(FULL_STATE)).runOn).toBe("export");
});
it("defaults a missing runOn to upload for other categories", () => {
const wire = toWirePolicy({ ...FULL_STATE, categoryId: "classification" });
delete (wire.output.options as Record<string, unknown>).runOn;
const wire = withNoStoredRunOn({
...FULL_STATE,
categoryId: "classification",
});
expect(fromWirePolicy(wire).runOn).toBe("upload");
});
@@ -109,6 +147,28 @@ describe("fromWirePolicy → round-trip", () => {
}
});
it("reads editor participation off the editor block, not sources", () => {
const wire = toWirePolicy(FULL_STATE);
expect(fromWirePolicy(wire).runsOnEditor).toBe(true);
expect(
fromWirePolicy({ ...wire, editor: { allowed: false, runOn: "upload" } })
.runsOnEditor,
).toBe(false);
});
it("prefers the editor block's moment over the legacy options bag", () => {
const wire = toWirePolicy(FULL_STATE);
wire.output.options.runOn = "upload";
wire.editor = { allowed: true, runOn: "export" };
expect(fromWirePolicy(wire).runOn).toBe("export");
});
it("falls back to the stored moment when the editor does not run it", () => {
const wire = toWirePolicy({ ...FULL_STATE, runOn: "export" });
wire.editor = { allowed: false, runOn: "upload" };
expect(fromWirePolicy(wire).runOn).toBe("export");
});
it("handles empty options gracefully", () => {
const decoded = fromWirePolicy({
id: "x",
@@ -120,6 +180,7 @@ describe("fromWirePolicy → round-trip", () => {
});
expect(decoded.categoryId).toBe("");
expect(decoded.sources).toEqual([]);
expect(decoded.runsOnEditor).toBe(false);
expect(decoded.runOn).toBe("upload");
expect(decoded.outputMode).toBe("new_version");
});
@@ -1,8 +1,8 @@
/**
* Bidirectional codec between the portal's frontend `PolicyDecodedState` and
* the backend `WirePolicy`. All policy-level metadata rides in
* `output.options`; `trigger` is always null (the editor fires runs on
* upload/export via `/run`). Mirrors the editor's `buildBackendPolicy` /
* `output.options`, including the `categoryId` the server reads back to find
* egress policies. Mirrors the editor's `buildBackendPolicy` /
* `fromBackendPolicy` from `policyPipeline.ts`, minus the editor-only
* `automation` blob and toolRegistry coupling.
*/
@@ -41,6 +41,9 @@ export function toWirePolicy(state: PolicyDecodedState): WirePolicy {
trigger: null,
steps: state.steps,
output: { type: "inline", options },
// Omitting this makes the backend stamp EditorConfig.disabled(), so a pause or a
// wizard save would quietly take the policy off the editor.
editor: { allowed: state.runsOnEditor, runOn: state.runOn },
};
}
@@ -63,10 +66,17 @@ export function fromWirePolicy(policy: WirePolicy): PolicyDecodedState {
enabled: policy.enabled,
categoryId,
sources: Array.isArray(raw.sources) ? raw.sources : [],
runsOnEditor: policy.editor?.allowed === true,
scopeTypes: Array.isArray(raw.scopeTypes) ? raw.scopeTypes : [],
reviewerEmail: str(raw.reviewerEmail),
fieldValues: raw.fieldValues ?? {},
runOn: resolveRunOn(raw.runOn, categoryId),
// The moment lives on `editor` now, but only carries meaning while the editor
// runs it (EditorConfig coerces a disabled policy's runOn to "upload"); fall back
// to the legacy options bag otherwise so the wizard still shows what was chosen.
runOn: resolveRunOn(
policy.editor?.allowed ? policy.editor.runOn : raw.runOn,
categoryId,
),
outputMode: raw.mode === "new_file" ? "new_file" : "new_version",
outputName: str(raw.name),
outputNamePosition: position,
@@ -6,7 +6,7 @@
* reviewer, fieldValues, runOn, output settings) inside `output.options` the
* same "options bag" the editor uses. `trigger` is always null for
* portal/editor-authored policies; the editor fires runs on upload/export via
* `/run`, so there is no server-side trigger.
* `/run`, and egress policies are found by their stored `categoryId`.
*/
// ── Wire types (match Policy.java / PipelineStep.java / PolicyRunView.java) ──
@@ -36,6 +36,16 @@ export interface WireOutputSpec {
options: Partial<WireOutputOptions>;
}
/**
* Mirrors `EditorConfig.java`. Absent only on records that never went through the
* backend (hand-built fixtures); a stored policy always carries it, derived from
* the legacy `output.options` bag when it predates the field.
*/
export interface WireEditorConfig {
allowed: boolean;
runOn: "upload" | "export";
}
export interface WirePolicy {
id: string;
name: string;
@@ -44,6 +54,7 @@ export interface WirePolicy {
trigger: null;
steps: WirePipelineStep[];
output: WireOutputSpec;
editor?: WireEditorConfig;
teamId?: string;
}
@@ -80,6 +91,12 @@ export interface PolicyDecodedState {
enabled: boolean;
categoryId: string;
sources: string[];
/**
* Whether the editor runs this policy per file. Its own field, not derived from
* `sources`: the seeded Classification policy is editor-run with empty sources,
* so re-deriving on write would silently take it off the editor.
*/
runsOnEditor: boolean;
scopeTypes: string[];
reviewerEmail: string;
fieldValues: Record<string, boolean | string | string[]>;
@@ -0,0 +1,135 @@
import { describe, it, expect, vi, beforeEach } from "vitest";
import {
decodedToState,
fetchPoliciesByCategory,
} from "@app/services/policyBackend";
const listPolicies = vi.fn();
vi.mock("@app/services/policyApi", () => ({
listPolicies: () => listPolicies(),
}));
/** A stored policy in the shape the backend returns. */
const policy = (
id: string,
categoryId?: string,
editor?: { allowed: boolean; runOn?: "upload" | "export" },
) => ({
id,
name: id,
enabled: true,
inputs: [],
steps: [{ operation: "/api/v1/misc/compress-pdf", parameters: {} }],
output: {
type: "inline",
options: { ...(categoryId ? { categoryId } : {}) },
},
outputIds: [],
editor: {
allowed: editor?.allowed ?? false,
runOn: editor?.runOn ?? ("upload" as const),
},
});
/** Decode one stored policy and project it onto the state the editor reads. */
async function stateOf(wire: ReturnType<typeof policy>, key: string) {
listPolicies.mockResolvedValue([wire]);
const decoded = (await fetchPoliciesByCategory()).get(key);
if (!decoded) throw new Error(`no decoded policy for ${key}`);
return decodedToState(decoded, undefined);
}
describe("fetchPoliciesByCategory", () => {
beforeEach(() => listPolicies.mockReset());
it("keys a catalogue policy by its category", async () => {
listPolicies.mockResolvedValue([
policy("pol-1", "classification", { allowed: true }),
]);
const map = await fetchPoliciesByCategory();
expect(map.get("classification")?.id).toBe("pol-1");
});
it("keeps a pipeline that has no category, keyed by its id", async () => {
// Built on the Pipelines page, so no category tile stamped it. It is still a policy: one set
// to run on editor uploads has to reach the editor's auto-run, which iterates this map.
listPolicies.mockResolvedValue([policy("pol-adhoc")]);
const map = await fetchPoliciesByCategory();
expect(map.has("pol-adhoc")).toBe(true);
expect(map.get("pol-adhoc")?.id).toBe("pol-adhoc");
});
it("carries both kinds at once without either displacing the other", async () => {
listPolicies.mockResolvedValue([
policy("pol-1", "classification", { allowed: true }),
policy("pol-adhoc"),
]);
const map = await fetchPoliciesByCategory();
expect([...map.keys()].sort()).toEqual(["classification", "pol-adhoc"]);
});
it("records run order from the list, which is the team's order", async () => {
listPolicies.mockResolvedValue([
policy("pol-1", "security", { allowed: true }),
policy("pol-adhoc"),
]);
const map = await fetchPoliciesByCategory();
expect(map.get("security")?.order).toBe(0);
expect(map.get("pol-adhoc")?.order).toBe(1);
});
});
describe("decodedToState — runsOnEditor", () => {
beforeEach(() => listPolicies.mockReset());
it("runs a catalogue tile that opted into the editor", async () => {
const state = await stateOf(
policy("pol-1", "security", { allowed: true }),
"security",
);
expect(state.runsOnEditor).toBe(true);
});
// Participation is the policy's own flag now, so a tile that never opted in does not run in the
// editor just because nobody narrowed its scope.
it("does not run a catalogue tile that never opted in", async () => {
const state = await stateOf(policy("pol-1", "security"), "security");
expect(state.runsOnEditor).toBe(false);
});
it("does not run a builder pipeline that never named the editor", async () => {
// Blank here means nothing stamped it - the tile default would fire an S3 or folder
// pipeline on every editor upload.
const state = await stateOf(policy("pol-adhoc"), "pol-adhoc");
expect(state.runsOnEditor).toBe(false);
});
it("runs a builder pipeline that names the editor outright", async () => {
const state = await stateOf(
policy("pol-adhoc", undefined, { allowed: true }),
"pol-adhoc",
);
expect(state.runsOnEditor).toBe(true);
});
it("marks only a catalogue tile as a built-in default", async () => {
expect(
(await stateOf(policy("pol-1", "security"), "security")).isDefault,
).toBe(true);
expect((await stateOf(policy("pol-adhoc"), "pol-adhoc")).isDefault).toBe(
false,
);
});
});
@@ -32,8 +32,10 @@ export async function fetchPoliciesByCategory(): Promise<
const byCategory = new Map<string, DecodedPolicy>();
stored.forEach((policy, index) => {
const decoded = fromBackendPolicy(policy);
if (decoded.categoryId)
byCategory.set(decoded.categoryId, { ...decoded, order: index });
// A pipeline built on the Pipelines page has no category tile, so it keys by its own id
// rather than being dropped - one set to run on the editor still has to reach the auto-run.
const key = decoded.categoryId || decoded.id;
if (key) byCategory.set(key, { ...decoded, order: index });
});
return byCategory;
}
@@ -50,7 +52,9 @@ export function decodedToState(
return {
configured: true,
status: decoded.enabled ? "active" : "paused",
name: decoded.name,
sources: decoded.sources,
runsOnEditor: decoded.runsOnEditor,
scopeTypes: decoded.scopeTypes,
reviewerEmail: decoded.reviewerEmail,
fieldValues: decoded.fieldValues,
@@ -62,8 +66,8 @@ export function decodedToState(
backendId: decoded.id,
// Server-side run-order position (team-wide); drives the settings reorder list.
order: decoded.order,
// Catalog-category policies are built-in defaults (not deletable).
isDefault: true,
// Catalog-category policies are built-in defaults (not deletable); a builder pipeline is not.
isDefault: Boolean(decoded.categoryId),
};
}
@@ -0,0 +1,97 @@
/**
* Fire a single backend policy run for one file and record it. Shared by the auto-run engine (which
* dispatches file-producing policies and their chain) and the classification policy (which dispatches
* its own AI escalation), so both take one bounded dispatch slot and record runs the same way.
*/
import { fileStorage } from "@app/services/fileStorage";
import {
runStoredPolicy,
resolvePolicyRunTarget,
} from "@app/services/policyApi";
import {
acquireDispatchSlot,
releaseDispatchSlot,
} from "@app/components/policies/dispatchSemaphore";
import {
markDispatched,
recordRunStart,
} from "@app/components/policies/policyRunStore";
import type { FileId } from "@app/types/file";
import type { StirlingFile } from "@app/types/fileContext";
/** Wait for an upload's bytes to land in IndexedDB (~5s): the stub surfaces in the
* file list before its bytes are committed, so an eager fetch would miss the file. */
const FILE_WAIT_TRIES = 20;
const FILE_WAIT_MS = 250;
const delay = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));
/** Resolve the file's bytes, fire a backend run, and record it. */
export async function runPolicyOnFile(
categoryId: string,
backendId: string,
fileId: FileId,
fileName: string,
// Chained (downstream) dispatch — jumps the dispatch queue so a file mid-chain
// finishes its flow before new files start (see acquireDispatchSlot).
priority = false,
): Promise<void> {
// A freshly-uploaded file's bytes are written to IndexedDB asynchronously, so
// its stub can appear in the file list a beat before getStirlingFile resolves
// it. Wait briefly rather than bail — and DON'T mark dispatched until we hold
// the file, or a too-early miss would skip enforcement on that file forever.
// (The caller's in-flight guard prevents double-dispatch during this wait.)
// A transient IndexedDB error is treated as a miss (not a throw), so it retries
// and then marks dispatched rather than rejecting into a hot re-dispatch loop.
const tryGetFile = async (): Promise<StirlingFile | null> => {
try {
return await fileStorage.getStirlingFile(fileId);
} catch {
return null;
}
};
let file = await tryGetFile();
for (let i = 0; i < FILE_WAIT_TRIES && !file; i++) {
await delay(FILE_WAIT_MS);
file = await tryGetFile();
}
if (!file) {
// File genuinely gone (removed before it could run) — mark so we don't loop.
markDispatched(categoryId, fileId);
return;
}
// Bounded upload window — see MAX_CONCURRENT_DISPATCHES. Only the POST is
// gated; the IDB wait above never holds a slot.
await acquireDispatchSlot(priority);
try {
const target = resolvePolicyRunTarget();
// Recorded against a document this browser can resolve. One file per run, which is the only
// shape the server keeps a reference for.
const runId = await runStoredPolicy(backendId, [file], fileId);
// recordRunStart marks this (policy, file) dispatched as it records the run.
recordRunStart({
runId,
categoryId,
fileId,
fileName,
fileSize: file.size,
target,
status: "PENDING",
outputs: [],
error: null,
startedAt: Date.now(),
});
} catch (err) {
// Dispatch failed (e.g. policy deleted/404 or backend offline). Mark dispatched so we don't hammer;
// the absent run simply won't appear in the activity feed. If the backend did
// start a run we never recorded, reconcileServerRuns rediscovers it.
console.debug(
`[PolicyAutoRun] Failed to dispatch policy ${categoryId} (${backendId}):`,
err,
);
markDispatched(categoryId, fileId);
} finally {
releaseDispatchSlot();
}
}
@@ -0,0 +1,129 @@
import { describe, it, expect, vi, beforeEach } from "vitest";
import type { PoliciesByCategory, PolicyState } from "@app/types/policies";
// Which policies export-time enforcement picks up: the policy's own editor flag, not its scope.
const loadPolicies = vi.fn<() => PoliciesByCategory>();
vi.mock("@app/services/policyStorage", () => ({
loadPolicies: () => loadPolicies(),
}));
const runStoredPolicy = vi.fn(async (_id: string) => "run-1");
vi.mock("@app/services/policyApi", () => ({
runStoredPolicy: (id: string) => runStoredPolicy(id),
// One output, so a run completes rather than throwing "produced no output" - which would abort
// the per-file policy loop after the first policy and hide the order under test.
getPolicyRun: async () => ({
status: "COMPLETED",
outputs: [{ fileId: "out-1", fileName: "doc.pdf" }],
}),
downloadPolicyOutput: async () => new Blob(),
resolvePolicyRunTarget: () => "local",
}));
vi.mock("@app/components/policies/policyRunStore", () => ({
recordRunStart: vi.fn(),
isDispatched: () => false,
}));
// Run the queued task inline: the queue's own behaviour is not under test here.
vi.mock("@app/components/policies/enforcementQueue", () => ({
runQueued: <T>(_meta: unknown, task: () => Promise<T>) => task(),
}));
vi.mock("@app/components/toast", () => ({
alert: () => "toast-1",
updateToast: vi.fn(),
dismissToast: vi.fn(),
}));
vi.mock("@app/i18n", () => ({ default: { t: (key: string) => key } }));
const { enforceExportPolicies } = await import("@app/services/policyExport");
/** An active export-time policy as the local store holds it. */
const exportPolicy = (over: Partial<PolicyState>): PolicyState =>
({
configured: true,
status: "active",
backendId: "backend-1",
sources: [],
runsOnEditor: false,
scopeTypes: [],
reviewerEmail: "",
fieldValues: {},
outputMode: "new_version",
outputName: "",
runOn: "export",
isDefault: false,
...over,
}) as PolicyState;
const pdf = () =>
new File(["%PDF-1.4"], "doc.pdf", { type: "application/pdf" });
describe("export-time policy selection", () => {
beforeEach(() => runStoredPolicy.mockClear());
it("enforces an editor pipeline set to run on export", async () => {
loadPolicies.mockReturnValue({
"builder-1": exportPolicy({
sources: ["editor"],
runsOnEditor: true,
backendId: "backend-editor",
}),
} as unknown as PoliciesByCategory);
await enforceExportPolicies([pdf()], ["file-1"]);
expect(runStoredPolicy).toHaveBeenCalledWith("backend-editor");
});
it("leaves a swept pipeline alone, even though its source list is blank", async () => {
loadPolicies.mockReturnValue({
"builder-2": exportPolicy({
sources: [],
runsOnEditor: false,
backendId: "backend-swept",
}),
} as unknown as PoliciesByCategory);
await enforceExportPolicies([pdf()], ["file-1"]);
expect(runStoredPolicy).not.toHaveBeenCalled();
});
it("still enforces a catalogue tile that nobody has narrowed", async () => {
loadPolicies.mockReturnValue({
security: exportPolicy({
sources: [],
// A tile is blank because it was never narrowed, so it does run on the editor.
runsOnEditor: true,
backendId: "backend-security",
}),
} as unknown as PoliciesByCategory);
await enforceExportPolicies([pdf()], ["file-1"]);
expect(runStoredPolicy).toHaveBeenCalledWith("backend-security");
});
it("enforces in the team's run order, not object order", async () => {
loadPolicies.mockReturnValue({
second: exportPolicy({
runsOnEditor: true,
backendId: "backend-second",
order: 1,
}),
first: exportPolicy({
runsOnEditor: true,
backendId: "backend-first",
order: 0,
}),
} as unknown as PoliciesByCategory);
await enforceExportPolicies([pdf()], ["file-1"]);
expect(runStoredPolicy.mock.calls.map(([id]) => id)).toEqual([
"backend-first",
"backend-second",
]);
});
});
@@ -62,22 +62,28 @@ function activeExportPolicies(): ExportPolicy[] {
const labels = new Map(
loadPolicyCatalog().categories.map((c) => [c.id, c.label]),
);
return Object.entries(loadPolicies())
.filter(
([, s]) =>
s.configured &&
s.status === "active" &&
s.backendId &&
(s.sources.length === 0 || s.sources.includes("editor")) &&
s.runOn === "export",
)
.map(([id, s]) => ({
categoryId: id,
backendId: s.backendId as string,
label: labels.get(id) ?? "Policy",
outputMode: s.outputMode === "new_file" ? "new_file" : "new_version",
accent: `var(--color-${ROW_ACCENT[id] ?? "blue"})`,
}));
return (
Object.entries(loadPolicies())
.filter(
([, s]) =>
s.configured &&
s.status === "active" &&
s.backendId &&
s.runsOnEditor &&
s.runOn === "export",
)
// Same team-wide run order the upload path uses: enforcement is not commutative (a watermark
// then a flatten is not a flatten then a watermark), so both paths must agree on the sequence.
.sort(([, a], [, b]) => (a.order ?? 0) - (b.order ?? 0))
.map(([id, s]) => ({
categoryId: id,
backendId: s.backendId as string,
// A builder pipeline has no built-in category, so it labels by its own name.
label: labels.get(id) ?? s.name ?? "Policy",
outputMode: s.outputMode === "new_file" ? "new_file" : "new_version",
accent: `var(--color-${ROW_ACCENT[id] ?? "blue"})`,
}))
);
}
/** Run one policy on a file and resolve the enforced bytes + run info (throws on
@@ -111,7 +111,8 @@ const samplePolicy = {
updatedAt: "",
},
pipelineSteps: [{ operation: "/api/v1/misc/compress-pdf", parameters: {} }],
sources: ["editor"],
sources: [],
runsOnEditor: true,
scopeTypes: ["Contracts"],
reviewerEmail: "me@x.com",
fieldValues: { minConfidence: "80%" },
@@ -147,7 +148,8 @@ describe("buildBackendPolicy", () => {
expect(decoded.id).toBe("p1");
expect(decoded.categoryId).toBe("security");
expect(decoded.enabled).toBe(true);
expect(decoded.sources).toEqual(["editor"]);
expect(decoded.sources).toEqual([]);
expect(decoded.runsOnEditor).toBe(true);
expect(decoded.scopeTypes).toEqual(["Contracts"]);
expect(decoded.reviewerEmail).toBe("me@x.com");
expect(decoded.fieldValues).toEqual({ minConfidence: "80%" });
@@ -11,7 +11,7 @@
* using that registry.
*/
import { resolveRunOn } from "@app/policies/runOn";
import { resolveRunOn, type PolicyRunOn } from "@app/policies/runOn";
import type { AutomationConfig } from "@app/types/automation";
import type { ToolRegistry } from "@app/data/toolsTaxonomy";
import type { PolicyFolderSettings } from "@app/types/policies";
@@ -56,6 +56,14 @@ export interface BackendPolicy {
trigger: BackendTriggerConfig | null;
steps: BackendPipelineStep[];
output: BackendOutputSpec;
/** Whether the editor runs this policy per file, and on which moment. */
editor?: BackendEditorConfig;
}
/** Mirrors the backend `EditorConfig`. */
export interface BackendEditorConfig {
allowed: boolean;
runOn: PolicyRunOn;
}
/**
@@ -217,6 +225,7 @@ export interface PolicyToStore {
*/
pipelineSteps: BackendPipelineStep[];
sources: string[];
runsOnEditor: boolean;
scopeTypes: string[];
reviewerEmail: string;
fieldValues: Record<string, boolean | string | string[]>;
@@ -233,6 +242,8 @@ export interface DecodedPolicy {
/** Null if the stored policy carried no automation blob. */
automation: AutomationConfig | null;
sources: string[];
/** Whether the editor runs this policy per file, straight from the policy's own flag. */
runsOnEditor: boolean;
scopeTypes: string[];
reviewerEmail: string;
fieldValues: Record<string, boolean | string | string[]>;
@@ -280,7 +291,6 @@ export function buildBackendPolicy(input: PolicyToStore): BackendPolicy {
maxRetries: input.folder.maxRetries,
retryDelayMinutes: input.folder.retryDelayMinutes,
automation: input.automation,
runOn: input.folder.runOn,
// Policy-level metadata (no trigger bag to hold it any more).
categoryId: input.categoryId,
sources: input.sources,
@@ -289,6 +299,10 @@ export function buildBackendPolicy(input: PolicyToStore): BackendPolicy {
fieldValues: input.fieldValues,
},
},
editor: {
allowed: input.runsOnEditor,
runOn: input.folder.runOn,
},
};
}
@@ -298,6 +312,7 @@ export function fromBackendPolicy(policy: BackendPolicy): DecodedPolicy {
// Metadata lives in output.options; legacy records kept it in trigger.options,
// so merge both (output wins) to decode either shape.
const meta = { ...(policy.trigger?.options ?? {}), ...output };
const editor = policy.editor;
const str = (v: unknown, fallback = "") =>
typeof v === "string" ? v : fallback;
const num = (v: unknown, fallback: number) =>
@@ -316,8 +331,9 @@ export function fromBackendPolicy(policy: BackendPolicy): DecodedPolicy {
reviewerEmail: str(meta.reviewerEmail),
fieldValues:
(meta.fieldValues as DecodedPolicy["fieldValues"] | undefined) ?? {},
runsOnEditor: editor?.allowed === true,
folder: {
runOn: resolveRunOn(meta.runOn, categoryId),
runOn: resolveRunOn(editor?.runOn, categoryId),
// Legacy/missing output.mode defaults to new_version, not new_file.
outputMode: output.mode === "new_file" ? "new_file" : "new_version",
outputName: str(output.name),
@@ -60,6 +60,43 @@ describe("policyStorage", () => {
expect(p.routing.configured).toBe(false);
});
it("migrates a pre-runsOnEditor row narrowed to non-editor sources off the editor", () => {
// Stored before runsOnEditor existed: no such field, sources exclude the editor.
localStorage.setItem(
"stirling-policies-state",
JSON.stringify({
security: { configured: true, status: "active", sources: ["s3"] },
}),
);
// Without the migration the default (true) would wrongly win.
expect(loadPolicies().security.runsOnEditor).toBe(false);
});
it("migrates a pre-runsOnEditor row listing the editor onto the editor", () => {
localStorage.setItem(
"stirling-policies-state",
JSON.stringify({
security: { configured: true, status: "active", sources: ["editor"] },
}),
);
expect(loadPolicies().security.runsOnEditor).toBe(true);
});
it("leaves an explicit runsOnEditor untouched", () => {
localStorage.setItem(
"stirling-policies-state",
JSON.stringify({
security: {
configured: true,
status: "active",
sources: ["editor"],
runsOnEditor: false,
},
}),
);
expect(loadPolicies().security.runsOnEditor).toBe(false);
});
it("fires a change event on update", () => {
const cb = vi.fn();
const off = onPoliciesChange(cb);
@@ -19,6 +19,7 @@ function defaultState(categoryId: string): PolicyState {
configured: false,
status: "default",
sources: ["editor"],
runsOnEditor: true,
scopeTypes: [],
// Empty by default; the wizard defaults the reviewer to the signed-in user.
reviewerEmail: "",
@@ -54,7 +55,14 @@ export function loadPolicies(): PoliciesByCategory {
// category gets a default rather than being undefined.
const out: PoliciesByCategory = {};
loadPolicyCatalog().categories.forEach((cat, index) => {
const merged = { ...defaultState(cat.id), ...(parsed[cat.id] ?? {}) };
const stored = parsed[cat.id];
const merged = { ...defaultState(cat.id), ...(stored ?? {}) };
// Migration: a row stored before runsOnEditor existed has no such field, so the default (true)
// would put a tile narrowed to non-editor sources on the editor until the first reconcile lands.
// Derive it from the legacy signal (the editor in its sources), mirroring the decode rule.
if (stored && stored.runsOnEditor === undefined) {
merged.runsOnEditor = (stored.sources ?? []).includes("editor");
}
// Migration: clear the obsolete persisted reviewer email so it re-defaults
// to the real signed-in user.
if (merged.reviewerEmail === STALE_REVIEWER_EMAIL)
@@ -64,6 +72,11 @@ export function loadPolicies(): PoliciesByCategory {
if (merged.order == null) merged.order = index;
out[cat.id] = merged;
});
// Builder pipelines key by their own id, so the walk above misses them. Carried through as
// stored: a tile's defaults would mark them built-in and put them on the editor uninvited.
for (const [key, state] of Object.entries(parsed)) {
if (!out[key] && state) out[key] = state as PolicyState;
}
return out;
}
@@ -118,6 +131,26 @@ export function reorderPolicies(
return next;
}
/**
* Drop cached entries entirely (no default seeded back). For builder pipelines the backend has
* deleted: keyed by their own id, they have no built-in category to fall back to, so a left-behind
* entry keeps a dead backendId that the auto-run still tries to dispatch. Built-in categories are
* never forgotten - they reseed on the next read anyway.
*/
export function forgetPolicies(ids: string[]): PoliciesByCategory {
const current = loadPolicies();
const catalogIds = new Set(loadPolicyCatalog().categories.map((c) => c.id));
const next: PoliciesByCategory = { ...current };
let removed = false;
for (const id of ids) {
if (catalogIds.has(id) || !(id in next)) continue;
delete next[id];
removed = true;
}
if (removed) persist(next);
return next;
}
/** Reset a category to its unconfigured default (the "Delete policy" action). */
export function resetPolicy(categoryId: string): PoliciesByCategory {
return updatePolicy(categoryId, {
@@ -120,6 +120,10 @@ export interface PolicyState {
status: PolicyStatus;
/** Selected sources (ids from POLICY_SOURCES). */
sources: string[];
/** The policy's own name. Set for builder pipelines, which have no built-in category label. */
name?: string;
/** Whether the policy runs in the editor as each file passes through (resolved at decode). */
runsOnEditor?: boolean;
/** When non-empty, narrows the policy to these document types. */
scopeTypes: string[];
/** Email that low-confidence enforcements are routed to. */
@@ -188,6 +192,7 @@ export interface PolicyWizardResult {
automation: AutomationConfig;
fieldValues: Record<string, boolean | string | string[]>;
sources: string[];
runsOnEditor: boolean;
scopeTypes: string[];
reviewerEmail: string;
/** Output + retry settings for the backing folder. */
@@ -224,6 +229,7 @@ export interface PolicyConfigResult {
unresolvedOps: string[];
fieldValues: Record<string, boolean | string | string[]>;
sources: string[];
runsOnEditor: boolean;
scopeTypes: string[];
reviewerEmail: string;
folder: PolicyFolderSettings;