Stirling-PDF/devGuide/STORAGE_ENCRYPTION_AT_REST.md

# Storage Encryption at Rest

Encrypts files stored by Stirling (My Files, workflow files) so the bytes on disk, in the database,
or in S3 are unreadable without the master key. Requires a Pro or Enterprise licence to enable.

> **Back up the master key.** Losing it makes every encrypted stored file permanently
> unrecoverable. There is no recovery path by design — that is what makes the encryption
> meaningful.

> **Audit trail requires Enterprise.** Encryption itself works on Pro, but the audit events
> below (encrypt, decrypt, revocation, plaintext export, migration) are only recorded on an
> Enterprise licence — the audit subsystem is Enterprise-gated platform-wide. On Pro the files
> are encrypted exactly the same way, but there is no access trail, which matters if you are
> enabling this to satisfy an audit-logging requirement (HIPAA, CMMC). A warning is logged at
> startup when encryption is enabled without an Enterprise licence.

## How it works

Envelope encryption, three levels:

| Level | What it is | Where it lives |
|---|---|---|
| Master key | Wraps the scope keys | Config property, env var, or `configs/file-encryption.key` |
| Scope key (KEK) | One per team; wraps each file's data key | `file_encryption_keys` table, master-key-wrapped |
| Data key (DEK) | One per stored blob; encrypts the bytes | Inside the blob's own header, scope-key-wrapped |

Each blob is self-describing: an `SPDFEAR1` header carries the format version, the scope key's id,
the plaintext length, and the wrapped data key, followed by AES-256-GCM streaming ciphertext
(1 MiB segments). The header is bound as associated data to both the key wrap and the payload, so a
header cannot be transplanted between blobs.

Consequences of that design worth knowing:

- Blobs without the magic prefix are treated as plaintext and passed through, so enabling the
  feature needs no migration and old files keep working.
- Because the key id is pinned per blob, moving a user between teams never breaks their existing
  files.
- Plaintext sizes are what get recorded in the database, so quotas and `Content-Length` are
  unaffected (ciphertext on disk is ~96 bytes + 16 bytes/MiB larger).
- Presigned S3 download URLs are suppressed once encrypted content can exist — a presigned GET
  would hand raw ciphertext to the browser — so those downloads stream through the application.

## Enabling it

```yaml
storage:
  encryption:
    enabled: true
```

The master key is resolved in this order:

1. `stirling.security.fileEncryptionKey` property
2. `STIRLING_FILE_ENCRYPTION_KEY` environment variable
3. an auto-generated `configs/file-encryption.key` (owner-only permissions)

Generate a key with:

```bash
openssl rand -base64 32
```

It must decode to exactly 32 bytes; anything else fails at startup rather than silently
downgrading the cipher. The startup log prints a fingerprint (a SHA-256 prefix, never the key) so
you can verify a backup matches the live key.

**Cluster mode** (`cluster.enabled=true`) requires the key to be set explicitly and identically on
every node; the auto-generated file is refused, because a node-local key would make files written
elsewhere unreadable.

### Turning it off

Disabling only stops encrypting *new* writes. Existing encrypted files stay readable as long as the
key material is present — the decrypt path is always active and is never licence-gated, so a lapsed
licence cannot lock you out of your own data.

## Encrypting files that already exist

Enabling the flag does not touch the existing plaintext backlog. To convert it:

```bash
curl -X POST  http://localhost:8080/api/v1/admin/storage-encryption/migrate
curl          http://localhost:8080/api/v1/admin/storage-encryption/migrate/status
```

The job is throttled, resumable, and safe to re-run: for each file it writes the encrypted copy
under a new storage key, swaps the database row only if nothing else changed it, and deletes the old
blob last. If a user replaces a file mid-migration their copy wins and the job skips it. Progress is
in-memory, so a restart mid-run loses the counters and `migrate/status` reports `IDLE` again — just
start it again; already-encrypted files are skipped. There is currently no way to cancel a run, and
on a cluster the guard is per-node, so trigger the migration on one node only.

## Revoking access (kill switch)

Disabling a scope key makes every file already stored under it fail closed with `403` until it is
re-enabled:

```bash
curl -X POST http://localhost:8080/api/v1/admin/storage-encryption/keys/{keyId}/disable
curl -X POST http://localhost:8080/api/v1/admin/storage-encryption/keys/{keyId}/enable
```

This revokes access to existing content; it does **not** stop the scope from storing new files. The
next upload finds no active key for the scope and mints one, so the team keeps working while its
history stays sealed. To stop new writes as well, turn encryption off (or take the scope's access
away at the application level) — the kill switch is aimed at stored bytes.

Because of that, re-enabling is status-aware: the key returns to `ACTIVE` if its scope has no other
active key, and to `RETIRED` if one was minted while it was revoked. Both statuses decrypt existing
content; only `ACTIVE` wraps new writes, so a scope never ends up with two keys competing for new
uploads. The `enable` response reports which status was applied.

This is reversible: the key material stays in the database and nothing is destroyed. No API path
deletes key material. On a cluster, other nodes pick the change up within their 60-second key-cache
window.

## Rotating the master key

Rotation only re-wraps the small `file_encryption_keys` table — file contents are never rewritten.

1. Set the new key as `stirling.security.fileEncryptionKey`.
2. Keep the outgoing key in `stirling.security.fileEncryptionKeyPrevious`.
3. Bump `stirling.security.fileEncryptionKeyVersion`.
4. Restart. Startup warns about rows still wrapped by the previous key. On a cluster, wait until
   **every** node carries both keys — a node still holding only the outgoing key cannot read a
   re-wrapped row, so rotating mid-deploy makes the lagging nodes fail on those scopes until they
   catch up.
5. `POST /api/v1/admin/storage-encryption/master/rotate`.
6. Confirm the response's `rewrapped` count and that `/status` shows every key row at the new
   `masterKeyVersion`.
7. Remove `fileEncryptionKeyPrevious` and restart.

**Do not skip step 6.** Until a row is re-wrapped it is still readable only with the outgoing key, so
removing that key while rows remain behind would seal the files under them. Startup verifies every
key row against the configured keys and refuses to start if any cannot be unwrapped, naming the count
and the first affected scope — so this shows up as a failed deploy, recoverable by putting the old key
back, rather than as unreadable files discovered later. Keep the outgoing key archived until a
restart has succeeded without it.

Key material is never accepted over HTTP; the endpoint only performs the re-wrap step.

## Auditing

**Requires an Enterprise licence** (see the note at the top): on Pro these events are silently
dropped by the audit subsystem, and a warning is logged at startup.

Encrypt, decrypt, denied-decrypt, key lifecycle, rotation, and migration events are written to the
audit trail, along with a `plaintextExport` marker whenever a plaintext copy of encrypted content is
served. Per-read decrypt events can be noisy on busy instances and can be turned off with
`storage.encryption.auditReads: false`; denials and key lifecycle events are always recorded.

Two semantics worth knowing when reading the trail:

- A `decrypt` event means a decryption was *authorised and opened*, not that bytes were read to
  completion — a load that is discarded still records one, and a re-read of the same open resource
  (e.g. an HTTP range request) does not record a second.
- `plaintextExport` is currently emitted for stored-file and share-link downloads. Workflow-file
  downloads are not yet marked.

## Status and backup verification

```bash
curl http://localhost:8080/api/v1/admin/storage-encryption/status
```

Reports whether writes are encrypted, the master-key fingerprint, encrypted vs plaintext file
counts, and every key row with its status history. All endpoints under
`/api/v1/admin/storage-encryption` require an admin account.

## What this protects against

Stolen disks, database dumps, exposed object-storage buckets, decommissioned media, and platform
users who are not authorised for a file. It is not a defence against an attacker who already has
root on a running instance — at that point the key is in memory. No storage-level encryption product
claims otherwise.