From df170fd4a6bbb76ceda91b5e86feec26b4ba9aab Mon Sep 17 00:00:00 2001 From: ConnorYoh <40631091+ConnorYoh@users.noreply.github.com> Date: Tue, 11 Aug 2026 13:01:58 +0100 Subject: [PATCH] =?UTF-8?q?feat(storage):=20encryption-at-rest=20ops=20?= =?UTF-8?q?=E2=80=94=20audit,=20admin=20kill=20switch,=20migration,=20key?= =?UTF-8?q?=20rotation=20(PR2)=20(#7173)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit # Description of Changes **PR2 of the encrypt-at-rest initiative — PR1 was #7155** Makes the P1 crypto operable and compliance-credible: admins can see the feature's state, flip the kill switch over an API instead of raw SQL, encrypt the pre-existing plaintext backlog, rotate the master key, and every security-relevant event lands in the audit trail. No frontend — that's PR3. **What was changed** - **Audit events** — new `STORAGE_ENCRYPTION` audit type, emitted through a small listener interface so the crypto classes stay plain objects: `encrypt`, `decrypt` (per-read events honour `storage.encryption.auditReads`, default **on** — HIPAA reviewers expect read audit; busy installs can disable), `decrypt.denied` (always), `key.created/disabled/enabled`, `master.rotated`, `migration.completed`, plus a `plaintextExport` marker whenever a plaintext copy of encrypted-at-rest content is served (with `inline` flag to distinguish in-app view from saved download). - **Admin API** `/api/v1/admin/storage-encryption` (`hasRole('ADMIN')`): - `GET /status` — write/decrypt state, **master-key fingerprint** (SHA-256 prefix for backup verification, never key material), encrypted vs plaintext file counts, full key list with status history. - `POST /keys/{id}/disable` / `enable` — the kill switch, now with active cache invalidation so revocation is immediate on the handling node (cross-node converges within the 60s cache TTL). Enable is restricted to DISABLED keys so two ACTIVE keys can't exist per scope. - `POST /migrate` + `GET /migrate/status` — encrypt-existing job. - `POST /master/rotate` — key material is never accepted over HTTP; keys come from config/env. - **Deliberately no delete endpoint** — key material can be disabled but never destroyed through the API. - **Encrypt-existing migration job** — new writes are encrypted from the moment the flag is on; this converts the backlog. Crash-safe per file: store the encrypted copy under a NEW storage key → compare-and-swap the DB row → only then delete the old blob. A CAS miss (user replaced the file mid-run) discards the job's copy — the user's file always wins. Worst crash outcome is an orphaned blob, never a lost file; re-runs are idempotent (`encryption_key_id IS NULL` selection, cursor-paged so failures can't wedge the loop). Handles all three blobs per row (main/history/audit-log), runs on a throttled virtual thread, single-flight guarded. - **Master-key rotation** — cheap by design thanks to the P1 hierarchy: rotate re-wraps the handful of KEK rows, zero file I/O. New config `stirling.security.fileEncryptionKeyPrevious` (+env) gives `unwrap` a fallback during rotation, and `stirling.security.fileEncryptionKeyVersion` marks which master wrapped each row. Runbook: set new key primary + old as previous + bump version → restart (startup self-check passes via fallback, warns about pending rows) → `POST /master/rotate` → remove the previous key. - **Shared state bean** — `StorageEncryptionState` is built once and shared by the storage decorator and the admin API, so kill-switch cache invalidation hits the same caches the decorator reads. **Reviewer notes** - The revoked→403 mapping promised for PR2 already landed in #7155 after manual testing; this PR adds the matching `decrypt.denied` audit event. - 19 new tests: audit emission (encrypt/decrypt/denied, legacy plaintext emits nothing), kill-switch immediacy (no TTL wait), rotation (previous-key fallback, re-wrap + cleanup, idempotent second call), migration (backlog encrypted byte-identical, CAS-miss discards own copy, per-file failure counting, concurrent-start rejection, write-disabled rejection), admin controller status/conflict/not-found paths. - Full proprietary suite: 2246/2247 green (the one failure is the pre-existing Windows-symlink FolderIdentitiesTest, unrelated). --- [ENCRYPTION_AT_REST_TEST_REPORT.html](https://github.com/user-attachments/files/30664158/ENCRYPTION_AT_REST_TEST_REPORT.html) --------- Co-authored-by: Reece Browne <74901996+reecebrowne@users.noreply.github.com> --- .../common/model/ApplicationProperties.java | 7 + .../src/main/resources/settings.yml.template | 25 +- .../proprietary/audit/AuditEventType.java | 3 + .../storage/config/StorageProviderConfig.java | 61 ++- .../controller/FileStorageController.java | 19 + .../StorageEncryptionAdminController.java | 275 +++++++++++ .../AuditingStorageEncryptionListener.java | 62 +++ .../crypto/EncryptingStorageProvider.java | 20 +- .../crypto/FileEncryptionKeyService.java | 234 +++++++-- .../crypto/FileEncryptionMasterKey.java | 66 ++- .../storage/crypto/ResourceUpload.java | 74 +++ .../StorageEncryptionAuditListener.java | 21 + .../crypto/StorageEncryptionErrors.java | 22 + .../crypto/StorageEncryptionState.java | 33 +- .../api/StorageEncryptionStatusResponse.java | 30 ++ .../FileEncryptionKeyRepository.java | 21 +- .../repository/StoredFileRepository.java | 50 ++ .../storage/service/FileStorageService.java | 8 +- .../StorageEncryptionMigrationService.java | 316 ++++++++++++ .../service/WorkflowSessionService.java | 28 +- .../config/StorageProviderConfigTest.java | 45 +- .../FileStorageControllerMoreTest.java | 4 +- .../controller/FileStorageControllerTest.java | 80 ++- ...rageEncryptionAdminControllerHttpTest.java | 209 ++++++++ .../StorageEncryptionAdminControllerTest.java | 228 +++++++++ ...AuditingStorageEncryptionListenerTest.java | 99 ++++ .../crypto/EncryptingStorageProviderTest.java | 61 ++- .../crypto/FileEncryptionKeyServiceTest.java | 157 ++++++ .../storage/crypto/InMemoryKeyRepo.java | 40 +- .../StoredFileMigrationQueriesDbTest.java | 148 ++++++ ...StorageEncryptionMigrationServiceTest.java | 467 ++++++++++++++++++ .../service/WorkflowSessionServiceTest.java | 43 ++ devGuide/README.md | 1 + devGuide/STORAGE_ENCRYPTION_AT_REST.md | 175 +++++++ 34 files changed, 3010 insertions(+), 122 deletions(-) create mode 100644 app/proprietary/src/main/java/stirling/software/proprietary/storage/controller/StorageEncryptionAdminController.java create mode 100644 app/proprietary/src/main/java/stirling/software/proprietary/storage/crypto/AuditingStorageEncryptionListener.java create mode 100644 app/proprietary/src/main/java/stirling/software/proprietary/storage/crypto/ResourceUpload.java create mode 100644 app/proprietary/src/main/java/stirling/software/proprietary/storage/crypto/StorageEncryptionAuditListener.java create mode 100644 app/proprietary/src/main/java/stirling/software/proprietary/storage/crypto/StorageEncryptionErrors.java create mode 100644 app/proprietary/src/main/java/stirling/software/proprietary/storage/model/api/StorageEncryptionStatusResponse.java create mode 100644 app/proprietary/src/main/java/stirling/software/proprietary/storage/service/StorageEncryptionMigrationService.java create mode 100644 app/proprietary/src/test/java/stirling/software/proprietary/storage/controller/StorageEncryptionAdminControllerHttpTest.java create mode 100644 app/proprietary/src/test/java/stirling/software/proprietary/storage/controller/StorageEncryptionAdminControllerTest.java create mode 100644 app/proprietary/src/test/java/stirling/software/proprietary/storage/crypto/AuditingStorageEncryptionListenerTest.java create mode 100644 app/proprietary/src/test/java/stirling/software/proprietary/storage/repository/StoredFileMigrationQueriesDbTest.java create mode 100644 app/proprietary/src/test/java/stirling/software/proprietary/storage/service/StorageEncryptionMigrationServiceTest.java create mode 100644 devGuide/STORAGE_ENCRYPTION_AT_REST.md diff --git a/app/common/src/main/java/stirling/software/common/model/ApplicationProperties.java b/app/common/src/main/java/stirling/software/common/model/ApplicationProperties.java index 99c8f65117..a9369519e5 100644 --- a/app/common/src/main/java/stirling/software/common/model/ApplicationProperties.java +++ b/app/common/src/main/java/stirling/software/common/model/ApplicationProperties.java @@ -1104,6 +1104,13 @@ public class ApplicationProperties { @Data public static class Encryption { private boolean enabled = false; + + /** + * Emit an audit event for every decrypt of an encrypted blob. Compliance reviewers + * (HIPAA) expect read audit, so it defaults on; busy multi-user installs can disable. + * Denied decrypts and key lifecycle events are always audited regardless. + */ + private boolean auditReads = true; } @Data diff --git a/app/core/src/main/resources/settings.yml.template b/app/core/src/main/resources/settings.yml.template index be93a4485f..4f7f62e084 100644 --- a/app/core/src/main/resources/settings.yml.template +++ b/app/core/src/main/resources/settings.yml.template @@ -292,26 +292,15 @@ storage: linkExpirationDays: 3 # Number of days before share links expire signing: enabled: false # set to 'true' to enable group signing workflow (requires storage.enabled) [ALPHA] - # ==================================================================================== - # ENCRYPTION AT REST - PRO / ENTERPRISE LICENSE REQUIRED TO ENABLE - # ==================================================================================== - # Encrypts stored files (AES-256 envelope encryption, per-team keys). The master key is - # resolved in this order: - # 1. stirling.security.fileEncryptionKey property - # 2. STIRLING_FILE_ENCRYPTION_KEY environment variable - # 3. an auto-generated configs/file-encryption.key (single-node only; cluster mode - # requires an explicitly shared key on every node) - # Generate a key with: openssl rand -base64 32 - # - # *** BACK UP THE MASTER KEY. Losing it makes every encrypted stored file *** - # *** permanently unrecoverable. Verify backups against the key fingerprint logged *** - # *** at startup. *** - # - # Enabling encrypts new writes only (existing files stay readable as plaintext). - # Disabling later only stops encrypting new writes - existing encrypted files remain - # readable as long as the key material is present. + # Encryption at rest for stored files (AES-256, per-team keys). Requires a Pro or + # Enterprise licence. Key setup, cluster requirements, the encrypt-existing migration, + # the revocation kill switch and master-key rotation are documented in + # devGuide/STORAGE_ENCRYPTION_AT_REST.md + # WARNING: back up the master key (configs/file-encryption.key by default) - losing it + # makes every encrypted stored file permanently unrecoverable. encryption: enabled: false # set to 'true' to encrypt stored files at rest + auditReads: true # audit every decrypt of an encrypted file (denied decrypts and key lifecycle events are always audited). NOTE: audit events require an Enterprise licence; encryption itself works on Pro. userListScope: org # Signing user-picker scope: 'org' (default) = whole instance, else caller's team only. autoPipeline: outputFolder: "" # Output folder for processed pipeline files (leave empty for default) diff --git a/app/proprietary/src/main/java/stirling/software/proprietary/audit/AuditEventType.java b/app/proprietary/src/main/java/stirling/software/proprietary/audit/AuditEventType.java index 433244a971..a1220a9fdd 100644 --- a/app/proprietary/src/main/java/stirling/software/proprietary/audit/AuditEventType.java +++ b/app/proprietary/src/main/java/stirling/software/proprietary/audit/AuditEventType.java @@ -19,6 +19,9 @@ public enum AuditEventType { // File operations - STANDARD level FILE_OPERATION("File operation"), + // Storage encryption at rest - STANDARD level + STORAGE_ENCRYPTION("Storage encryption operation"), + // PDF operations - STANDARD level PDF_PROCESS("PDF processing operation"), diff --git a/app/proprietary/src/main/java/stirling/software/proprietary/storage/config/StorageProviderConfig.java b/app/proprietary/src/main/java/stirling/software/proprietary/storage/config/StorageProviderConfig.java index 24814bfc69..1217d81cc8 100644 --- a/app/proprietary/src/main/java/stirling/software/proprietary/storage/config/StorageProviderConfig.java +++ b/app/proprietary/src/main/java/stirling/software/proprietary/storage/config/StorageProviderConfig.java @@ -21,10 +21,14 @@ import stirling.software.common.configuration.InstallationPathConfig; import stirling.software.common.model.ApplicationProperties; import stirling.software.common.util.TempFileManager; import stirling.software.proprietary.cluster.s3.S3Clients; +import stirling.software.proprietary.security.configuration.ee.KeygenLicenseVerifier.License; import stirling.software.proprietary.security.configuration.ee.LicenseKeyChecker; +import stirling.software.proprietary.service.AuditService; +import stirling.software.proprietary.storage.crypto.AuditingStorageEncryptionListener; import stirling.software.proprietary.storage.crypto.EncryptingStorageProvider; import stirling.software.proprietary.storage.crypto.FileEncryptionKeyService; import stirling.software.proprietary.storage.crypto.FileEncryptionMasterKey; +import stirling.software.proprietary.storage.crypto.StorageEncryptionAuditListener; import stirling.software.proprietary.storage.crypto.StorageEncryptionState; import stirling.software.proprietary.storage.provider.DatabaseStorageProvider; import stirling.software.proprietary.storage.provider.LocalStorageProvider; @@ -42,23 +46,33 @@ public class StorageProviderConfig { private final StoredFileBlobRepository storedFileBlobRepository; private final FileEncryptionKeyRepository fileEncryptionKeyRepository; private final LicenseKeyChecker licenseKeyChecker; + private final AuditService auditService; /** - * The encryption state behind the always-installed decorator. Key machinery is created eagerly - * when the write flag is on (licence-gated) or key rows already exist — so a wrong master key - * fails startup, not the first download — and lazily if encrypted content shows up later - * (config drift on one cluster node must fail loudly, never stream ciphertext). Turning the - * flag off or losing the licence only stops encrypting new writes; decryption stays available. + * The encryption state behind the always-installed decorator, shared with the admin API and + * migration job. Key machinery is created eagerly when the write flag is on (licence-gated) or + * key rows already exist — so a wrong master key fails startup, not the first download — and + * lazily if encrypted content shows up later (config drift on one cluster node must fail + * loudly, never stream ciphertext). Turning the flag off or losing the licence only stops + * encrypting new writes; decryption stays available. */ @Bean public StorageEncryptionState storageEncryptionState( @Value("${stirling.security.fileEncryptionKey:}") String configuredFileEncryptionKey, + @Value("${stirling.security.fileEncryptionKeyPrevious:}") + String previousFileEncryptionKey, + @Value("${stirling.security.fileEncryptionKeyVersion:1}") int fileEncryptionKeyVersion, @Value("${cluster.enabled:false}") boolean clusterEnabled, PlatformTransactionManager transactionManager) { boolean writeEnabled = applicationProperties.getStorage().getEncryption().isEnabled(); if (writeEnabled) { licenseKeyChecker.requireProOrEnterprise("storage.encryption"); + warnIfAuditUnavailable(); } + StorageEncryptionAuditListener listener = + new AuditingStorageEncryptionListener( + auditService, + applicationProperties.getStorage().getEncryption().isAuditReads()); // Key creation must commit independently of any caller transaction (see // FileEncryptionKeyService#createActive). TransactionTemplate requiresNew = new TransactionTemplate(transactionManager); @@ -68,8 +82,14 @@ public class StorageProviderConfig { writeEnabled, () -> createKeyService( - configuredFileEncryptionKey, clusterEnabled, requiresNew), - fileEncryptionKeyRepository); + configuredFileEncryptionKey, + previousFileEncryptionKey, + fileEncryptionKeyVersion, + clusterEnabled, + listener, + requiresNew), + fileEncryptionKeyRepository, + listener); // The registry table may not exist when storage is unused, so only probe if it is on. boolean probeForExistingKeys = !writeEnabled && applicationProperties.getStorage().isEnabled(); @@ -82,12 +102,33 @@ public class StorageProviderConfig { return state; } + /** + * Encryption at rest is available on Pro, but {@code AuditService} only records events on an + * Enterprise licence. Without this warning a Pro operator would enable encryption, be told it + * is audited, and silently get no encrypt/decrypt/revocation trail at all. + */ + private void warnIfAuditUnavailable() { + if (licenseKeyChecker.getPremiumLicenseEnabledResult() != License.ENTERPRISE) { + log.warn( + "Storage encryption at rest is enabled, but audit events require an Enterprise" + + " licence: encrypt/decrypt, revocation and plaintext-export events" + + " will NOT be recorded on this licence tier. Encryption itself is" + + " unaffected. See devGuide/STORAGE_ENCRYPTION_AT_REST.md"); + } + } + private FileEncryptionKeyService createKeyService( - String configuredKey, boolean clusterEnabled, TransactionOperations keyCreationTx) { + String configuredKey, + String previousKey, + int keyVersion, + boolean clusterEnabled, + StorageEncryptionAuditListener listener, + TransactionOperations keyCreationTx) { FileEncryptionMasterKey masterKey = - new FileEncryptionMasterKey(configuredKey, clusterEnabled); + new FileEncryptionMasterKey(configuredKey, previousKey, keyVersion, clusterEnabled); FileEncryptionKeyService keyService = - new FileEncryptionKeyService(fileEncryptionKeyRepository, masterKey, keyCreationTx); + new FileEncryptionKeyService( + fileEncryptionKeyRepository, masterKey, listener, keyCreationTx); // Wrong key must fail fast, not silently start a second key hierarchy. keyService.verifyMasterKey(); return keyService; diff --git a/app/proprietary/src/main/java/stirling/software/proprietary/storage/controller/FileStorageController.java b/app/proprietary/src/main/java/stirling/software/proprietary/storage/controller/FileStorageController.java index 4eb299cd24..3e5589b31d 100644 --- a/app/proprietary/src/main/java/stirling/software/proprietary/storage/controller/FileStorageController.java +++ b/app/proprietary/src/main/java/stirling/software/proprietary/storage/controller/FileStorageController.java @@ -5,6 +5,7 @@ import java.net.URI; import java.time.Duration; import java.util.List; import java.util.Locale; +import java.util.Map; import java.util.Optional; import org.springframework.http.ContentDisposition; @@ -31,7 +32,9 @@ import io.swagger.v3.oas.annotations.tags.Tag; import lombok.RequiredArgsConstructor; import lombok.extern.slf4j.Slf4j; +import stirling.software.proprietary.audit.AuditEventType; import stirling.software.proprietary.security.model.User; +import stirling.software.proprietary.service.AuditService; import stirling.software.proprietary.storage.model.FileShare; import stirling.software.proprietary.storage.model.StoredFile; import stirling.software.proprietary.storage.model.api.CreateShareLinkRequest; @@ -56,6 +59,7 @@ public class FileStorageController { private final FileStorageService fileStorageService; private final StorageProvider storageProvider; + private final AuditService auditService; @PostMapping( value = "/files", @@ -262,6 +266,21 @@ public class FileStorageController { private ResponseEntity buildFileResponse( StoredFile file, boolean inline) { org.springframework.core.io.Resource resource = fileStorageService.loadFile(file); + if (file.getEncryptionKeyId() != null) { + // Compliance marker: a plaintext copy of encrypted-at-rest content left the platform + // (inline=true is an in-app view; false is a saved download). + auditService.audit( + AuditEventType.STORAGE_ENCRYPTION, + Map.of( + "action", + "plaintextExport", + "fileId", + file.getId(), + "inline", + inline, + "keyId", + file.getEncryptionKeyId())); + } String contentType = file.getContentType() == null ? MediaType.APPLICATION_OCTET_STREAM_VALUE diff --git a/app/proprietary/src/main/java/stirling/software/proprietary/storage/controller/StorageEncryptionAdminController.java b/app/proprietary/src/main/java/stirling/software/proprietary/storage/controller/StorageEncryptionAdminController.java new file mode 100644 index 0000000000..d0845678ef --- /dev/null +++ b/app/proprietary/src/main/java/stirling/software/proprietary/storage/controller/StorageEncryptionAdminController.java @@ -0,0 +1,275 @@ +package stirling.software.proprietary.storage.controller; + +import java.time.Instant; +import java.util.List; +import java.util.Map; +import java.util.UUID; + +import org.springframework.dao.DataAccessException; +import org.springframework.data.domain.Sort; +import org.springframework.http.HttpStatus; +import org.springframework.http.MediaType; +import org.springframework.security.access.prepost.PreAuthorize; +import org.springframework.security.core.Authentication; +import org.springframework.security.core.context.SecurityContextHolder; +import org.springframework.web.bind.annotation.GetMapping; +import org.springframework.web.bind.annotation.PathVariable; +import org.springframework.web.bind.annotation.PostMapping; +import org.springframework.web.bind.annotation.RequestMapping; +import org.springframework.web.bind.annotation.RestController; +import org.springframework.web.server.ResponseStatusException; + +import io.swagger.v3.oas.annotations.tags.Tag; + +import lombok.RequiredArgsConstructor; +import lombok.extern.slf4j.Slf4j; + +import stirling.software.common.model.ApplicationProperties; +import stirling.software.proprietary.audit.AuditEventType; +import stirling.software.proprietary.service.AuditService; +import stirling.software.proprietary.storage.crypto.FileEncryptionKeyService; +import stirling.software.proprietary.storage.crypto.StorageEncryptionException; +import stirling.software.proprietary.storage.crypto.StorageEncryptionState; +import stirling.software.proprietary.storage.model.FileEncryptionKey; +import stirling.software.proprietary.storage.model.api.StorageEncryptionStatusResponse; +import stirling.software.proprietary.storage.repository.FileEncryptionKeyRepository; +import stirling.software.proprietary.storage.repository.StoredFileRepository; +import stirling.software.proprietary.storage.service.StorageEncryptionMigrationService; + +/** + * Admin surface for storage encryption at rest: status/backup verification, the per-scope kill + * switch, the encrypt-existing migration, and master-key rotation. Deliberately no delete endpoint + * — key material can be disabled but never destroyed through the API. + */ +@RestController +@RequestMapping("/api/v1/admin/storage-encryption") +@PreAuthorize("hasRole('ADMIN')") +@RequiredArgsConstructor +@Slf4j +@Tag(name = "Admin: Storage Encryption", description = "Encryption-at-rest administration") +public class StorageEncryptionAdminController { + + private final ApplicationProperties applicationProperties; + private final StorageEncryptionState encryptionState; + private final FileEncryptionKeyRepository keyRepository; + private final StoredFileRepository storedFileRepository; + private final StorageEncryptionMigrationService migrationService; + private final AuditService auditService; + + /** + * Reports write state, master-key fingerprint and encrypted/plaintext counts. Refuses rather + * than reading the registry when storage is off, because a deployment that never stores files + * may not have the table at all — the same reason the decorator's boot probe is gated. + */ + @GetMapping(value = "/status", produces = MediaType.APPLICATION_JSON_VALUE) + public StorageEncryptionStatusResponse status() { + if (!applicationProperties.getStorage().isEnabled()) { + throw new ResponseStatusException(HttpStatus.FORBIDDEN, "Storage is disabled"); + } + try { + return buildStatus(); + } catch (DataAccessException e) { + log.warn("Could not read storage encryption status: {}", e.getMessage()); + throw new ResponseStatusException( + HttpStatus.SERVICE_UNAVAILABLE, + "The storage encryption key registry could not be read", + e); + } + } + + private StorageEncryptionStatusResponse buildStatus() { + List keys = + keyRepository.findAll(Sort.by("createdAt")).stream() + .map( + k -> + new StorageEncryptionStatusResponse.KeyInfo( + k.getKeyId(), + k.getScopeType().name(), + k.getScopeId(), + k.getKeyVersion(), + k.getMasterKeyVersion(), + k.getStatus().name(), + k.getCreatedAt(), + k.getStatusChangedAt(), + k.getStatusChangedBy())) + .toList(); + String fingerprint = null; + Integer masterKeyVersion = null; + if (encryptionState.isMaterialised()) { + try { + FileEncryptionKeyService keyService = encryptionState.keyService(); + fingerprint = keyService.masterKey().fingerprint(); + masterKeyVersion = keyService.masterKey().currentVersion(); + } catch (StorageEncryptionException ignored) { + // Materialisation failed; status still reports counts and key rows. + } + } + return new StorageEncryptionStatusResponse( + encryptionState.isWriteEnabled(), + encryptionState.isMaterialised(), + fingerprint, + masterKeyVersion, + storedFileRepository.countByEncryptionKeyIdIsNotNull(), + storedFileRepository.countByEncryptionKeyIdIsNull(), + keys); + } + + /** Kill switch: content under this key fails closed (403) until re-enabled. Reversible. */ + @PostMapping("/keys/{keyId}/disable") + public StorageEncryptionStatusResponse.KeyInfo disableKey(@PathVariable UUID keyId) { + FileEncryptionKey row = setStatus(keyId, FileEncryptionKey.Status.DISABLED); + auditService.audit( + AuditEventType.STORAGE_ENCRYPTION, + Map.of("action", "key.disabled", "keyId", keyId.toString())); + return toKeyInfo(row); + } + + /** + * Reverses the kill switch: only DISABLED keys can be enabled, and they come back ACTIVE unless + * the scope acquired another active key while revoked, in which case they come back RETIRED — + * readable, but not a second key wrapping new writes. The response carries the resulting + * status. + */ + @PostMapping("/keys/{keyId}/enable") + public StorageEncryptionStatusResponse.KeyInfo enableKey(@PathVariable UUID keyId) { + FileEncryptionKey existing = + keyRepository + .findById(keyId) + .orElseThrow( + () -> + new ResponseStatusException( + HttpStatus.NOT_FOUND, "No such encryption key")); + if (existing.getStatus() != FileEncryptionKey.Status.DISABLED) { + throw new ResponseStatusException( + HttpStatus.CONFLICT, + "Key is not disabled (status: " + existing.getStatus() + ")"); + } + FileEncryptionKeyService keyService = requireKeyService(); + FileEncryptionKey row; + try { + row = keyService.enable(keyId, currentUsername()); + } catch (StorageEncryptionException e) { + throw new ResponseStatusException(HttpStatus.NOT_FOUND, e.getMessage()); + } + auditService.audit( + AuditEventType.STORAGE_ENCRYPTION, + Map.of( + "action", + "key.enabled", + "keyId", + keyId.toString(), + "status", + row.getStatus().name())); + return toKeyInfo(row); + } + + @PostMapping("/migrate") + public MigrationStatusResponse startMigration() { + try { + return MigrationStatusResponse.from(migrationService.start()); + } catch (IllegalStateException e) { + throw new ResponseStatusException(HttpStatus.CONFLICT, e.getMessage()); + } + } + + @GetMapping(value = "/migrate/status", produces = MediaType.APPLICATION_JSON_VALUE) + public MigrationStatusResponse migrationStatus() { + return migrationService + .status() + .map(MigrationStatusResponse::from) + .orElse(MigrationStatusResponse.IDLE); + } + + /** + * Re-wraps KEK rows below the configured master-key version under the primary master key. Key + * material is never accepted over HTTP — new keys arrive via config/env and a restart; this + * endpoint only performs the re-wrap step of the rotation runbook. + */ + @PostMapping("/master/rotate") + public Map rotateMasterKey() { + FileEncryptionKeyService keyService = requireKeyService(); + int rewrapped; + try { + rewrapped = keyService.rotateMasterKey(); + } catch (StorageEncryptionException e) { + throw new ResponseStatusException( + HttpStatus.INTERNAL_SERVER_ERROR, "Rotation failed: " + e.getMessage(), e); + } + auditService.audit( + AuditEventType.STORAGE_ENCRYPTION, + Map.of( + "action", + "master.rotated", + "rowsRewrapped", + rewrapped, + "masterKeyVersion", + keyService.masterKey().currentVersion())); + return Map.of( + "rewrapped", + rewrapped, + "masterKeyVersion", + keyService.masterKey().currentVersion()); + } + + private FileEncryptionKey setStatus(UUID keyId, FileEncryptionKey.Status status) { + FileEncryptionKeyService keyService = requireKeyService(); + try { + return keyService.setKeyStatus(keyId, status, currentUsername()); + } catch (StorageEncryptionException e) { + throw new ResponseStatusException(HttpStatus.NOT_FOUND, e.getMessage()); + } + } + + private FileEncryptionKeyService requireKeyService() { + try { + return encryptionState.keyService(); + } catch (StorageEncryptionException e) { + throw new ResponseStatusException( + HttpStatus.CONFLICT, + "Storage encryption is not configured: " + e.getMessage(), + e); + } + } + + private static String currentUsername() { + Authentication auth = SecurityContextHolder.getContext().getAuthentication(); + return auth != null ? auth.getName() : "system"; + } + + private static StorageEncryptionStatusResponse.KeyInfo toKeyInfo(FileEncryptionKey k) { + return new StorageEncryptionStatusResponse.KeyInfo( + k.getKeyId(), + k.getScopeType().name(), + k.getScopeId(), + k.getKeyVersion(), + k.getMasterKeyVersion(), + k.getStatus().name(), + k.getCreatedAt(), + k.getStatusChangedAt(), + k.getStatusChangedBy()); + } + + public record MigrationStatusResponse( + String state, + Long total, + Long processed, + Long skipped, + Long failed, + Instant startedAt, + Instant finishedAt) { + + static final MigrationStatusResponse IDLE = + new MigrationStatusResponse("IDLE", null, null, null, null, null, null); + + static MigrationStatusResponse from(StorageEncryptionMigrationService.MigrationStatus s) { + return new MigrationStatusResponse( + s.state().name(), + s.total(), + s.processed(), + s.skipped(), + s.failed(), + s.startedAt(), + s.finishedAt()); + } + } +} diff --git a/app/proprietary/src/main/java/stirling/software/proprietary/storage/crypto/AuditingStorageEncryptionListener.java b/app/proprietary/src/main/java/stirling/software/proprietary/storage/crypto/AuditingStorageEncryptionListener.java new file mode 100644 index 0000000000..2b46c9dc62 --- /dev/null +++ b/app/proprietary/src/main/java/stirling/software/proprietary/storage/crypto/AuditingStorageEncryptionListener.java @@ -0,0 +1,62 @@ +package stirling.software.proprietary.storage.crypto; + +import java.util.Map; +import java.util.UUID; + +import stirling.software.proprietary.audit.AuditEventType; +import stirling.software.proprietary.service.AuditService; + +/** + * Bridges storage-encryption events into the audit trail. Per-read decrypt events can be noisy on + * busy installs, so they honour {@code storage.encryption.auditReads}; denials and key lifecycle + * events are always recorded. + */ +public class AuditingStorageEncryptionListener implements StorageEncryptionAuditListener { + + private final AuditService auditService; + private final boolean auditReads; + + public AuditingStorageEncryptionListener(AuditService auditService, boolean auditReads) { + this.auditService = auditService; + this.auditReads = auditReads; + } + + @Override + public void encrypted(String storageKey, UUID keyId) { + auditService.audit( + AuditEventType.STORAGE_ENCRYPTION, + Map.of("action", "encrypt", "storageKey", storageKey, "keyId", keyId.toString())); + } + + @Override + public void decrypted(String storageKey, UUID keyId) { + if (!auditReads) { + return; + } + auditService.audit( + AuditEventType.STORAGE_ENCRYPTION, + Map.of("action", "decrypt", "storageKey", storageKey, "keyId", keyId.toString())); + } + + @Override + public void decryptDenied(UUID keyId, String reason) { + auditService.audit( + AuditEventType.STORAGE_ENCRYPTION, + Map.of("action", "decrypt.denied", "keyId", keyId.toString(), "reason", reason)); + } + + @Override + public void keyCreated(UUID keyId, String scope, int version) { + auditService.audit( + AuditEventType.STORAGE_ENCRYPTION, + Map.of( + "action", + "key.created", + "keyId", + keyId.toString(), + "scope", + scope, + "keyVersion", + version)); + } +} diff --git a/app/proprietary/src/main/java/stirling/software/proprietary/storage/crypto/EncryptingStorageProvider.java b/app/proprietary/src/main/java/stirling/software/proprietary/storage/crypto/EncryptingStorageProvider.java index 9a22dfd35d..9d1def1935 100644 --- a/app/proprietary/src/main/java/stirling/software/proprietary/storage/crypto/EncryptingStorageProvider.java +++ b/app/proprietary/src/main/java/stirling/software/proprietary/storage/crypto/EncryptingStorageProvider.java @@ -75,6 +75,15 @@ public class EncryptingStorageProvider implements StorageProvider { this(delegate, StorageEncryptionState.of(writeEnabled, keys), null); } + /** Test convenience with an explicit audit listener. */ + public EncryptingStorageProvider( + StorageProvider delegate, + FileEncryptionKeyService keys, + boolean writeEnabled, + StorageEncryptionAuditListener auditListener) { + this(delegate, StorageEncryptionState.of(writeEnabled, keys, auditListener), null); + } + @Override public StoredObject store(User owner, MultipartFile file) throws IOException { if (!state.isWriteEnabled()) { @@ -119,6 +128,7 @@ public class EncryptingStorageProvider implements StorageProvider { stored.getStorageKey(), kek.keyId(), file.getSize()); + state.auditListener().encrypted(stored.getStorageKey(), kek.keyId()); return stored.toBuilder() .sizeBytes(file.getSize()) .encryptionKeyId(kek.keyId().toString()) @@ -140,9 +150,9 @@ public class EncryptingStorageProvider implements StorageProvider { public Resource load(String storageKey) throws IOException { Resource raw = delegate.load(storageKey); if (raw.isOpen()) { - return wrapOneShot(raw); + return wrapOneShot(storageKey, raw); } - return wrapReopenable(raw); + return wrapReopenable(storageKey, raw); } @Override @@ -261,7 +271,7 @@ public class EncryptingStorageProvider implements StorageProvider { // ---- load helpers -------------------------------------------------------------------- /** Re-openable delegate (local file, DB byte array): sniff via a throwaway stream. */ - private Resource wrapReopenable(Resource raw) throws IOException { + private Resource wrapReopenable(String storageKey, Resource raw) throws IOException { byte[] prefix; try (InputStream in = raw.getInputStream()) { prefix = in.readNBytes(EncryptedFileFormat.HEADER_LENGTH); @@ -271,6 +281,7 @@ public class EncryptingStorageProvider implements StorageProvider { return raw; } byte[] dek = unwrapDek(header); + state.auditListener().decrypted(storageKey, header.keyId()); return new ReopenableDecryptedResource(raw, header, dek); } @@ -280,7 +291,7 @@ public class EncryptingStorageProvider implements StorageProvider { * tampered wrap) must close it — leaking here would starve the S3 connection pool precisely * when the kill switch is being exercised. */ - private Resource wrapOneShot(Resource raw) throws IOException { + private Resource wrapOneShot(String storageKey, Resource raw) throws IOException { InputStream in = raw.getInputStream(); try { byte[] prefix = in.readNBytes(EncryptedFileFormat.HEADER_LENGTH); @@ -306,6 +317,7 @@ public class EncryptingStorageProvider implements StorageProvider { } catch (GeneralSecurityException e) { throw new StorageEncryptionException("Failed to open decrypting stream", e); } + state.auditListener().decrypted(storageKey, header.keyId()); return new OneShotResource(decrypting, header.plaintextLength(), raw.getDescription()); } catch (IOException | RuntimeException e) { try { diff --git a/app/proprietary/src/main/java/stirling/software/proprietary/storage/crypto/FileEncryptionKeyService.java b/app/proprietary/src/main/java/stirling/software/proprietary/storage/crypto/FileEncryptionKeyService.java index 716d4f1a7b..c81f03249c 100644 --- a/app/proprietary/src/main/java/stirling/software/proprietary/storage/crypto/FileEncryptionKeyService.java +++ b/app/proprietary/src/main/java/stirling/software/proprietary/storage/crypto/FileEncryptionKeyService.java @@ -4,7 +4,11 @@ import java.nio.charset.StandardCharsets; import java.security.GeneralSecurityException; import java.security.SecureRandom; import java.time.Duration; +import java.time.LocalDateTime; +import java.util.ArrayList; import java.util.Base64; +import java.util.List; +import java.util.Optional; import java.util.UUID; import org.springframework.dao.DataIntegrityViolationException; @@ -37,6 +41,7 @@ public class FileEncryptionKeyService { private final FileEncryptionKeyRepository repository; private final FileEncryptionMasterKey masterKey; + private final StorageEncryptionAuditListener auditListener; /** * Runs key-row creation in its own committed transaction (REQUIRES_NEW in production). Callers @@ -54,18 +59,42 @@ public class FileEncryptionKeyService { public FileEncryptionKeyService( FileEncryptionKeyRepository repository, FileEncryptionMasterKey masterKey) { - this(repository, masterKey, TransactionOperations.withoutTransaction()); + this( + repository, + masterKey, + StorageEncryptionAuditListener.NOOP, + TransactionOperations.withoutTransaction()); + } + + public FileEncryptionKeyService( + FileEncryptionKeyRepository repository, + FileEncryptionMasterKey masterKey, + StorageEncryptionAuditListener auditListener) { + this(repository, masterKey, auditListener, TransactionOperations.withoutTransaction()); } public FileEncryptionKeyService( FileEncryptionKeyRepository repository, FileEncryptionMasterKey masterKey, TransactionOperations keyCreationTx) { + this(repository, masterKey, StorageEncryptionAuditListener.NOOP, keyCreationTx); + } + + public FileEncryptionKeyService( + FileEncryptionKeyRepository repository, + FileEncryptionMasterKey masterKey, + StorageEncryptionAuditListener auditListener, + TransactionOperations keyCreationTx) { this.repository = repository; this.masterKey = masterKey; + this.auditListener = auditListener; this.keyCreationTx = keyCreationTx; } + public FileEncryptionMasterKey masterKey() { + return masterKey; + } + public record ScopeKek(UUID keyId, byte[] key) {} /** The ACTIVE KEK for the owner's scope, created on first use. */ @@ -100,14 +129,17 @@ public class FileEncryptionKeyService { repository .findById(keyId) .orElseThrow( - () -> - new StorageEncryptionException( - "No encryption key " - + keyId - + " — the key registry does not match the" - + " stored data (restored from an older" - + " database backup?)")); + () -> { + auditListener.decryptDenied(keyId, "key not found"); + return new StorageEncryptionException( + "No encryption key " + + keyId + + " — the key registry does not match the" + + " stored data (restored from an older" + + " database backup?)"); + }); if (row.getStatus() == FileEncryptionKey.Status.DISABLED) { + auditListener.decryptDenied(keyId, "key disabled"); throw new StorageKeyRevokedException( "Encryption key " + keyId + " is disabled; access to this content is revoked"); } @@ -115,42 +147,164 @@ public class FileEncryptionKeyService { } /** - * Startup self-check: proves the resolved master key can unwrap an existing row, so a wrong key - * fails fast instead of silently writing new files under a second key hierarchy. + * Flips a key's status (the kill switch) and invalidates the local caches so the change is + * immediate on this node; other cluster nodes converge within the cache TTL. + */ + public FileEncryptionKey setKeyStatus(UUID keyId, FileEncryptionKey.Status status, String actor) + throws StorageEncryptionException { + FileEncryptionKey row = + repository + .findById(keyId) + .orElseThrow( + () -> new StorageEncryptionException("No encryption key " + keyId)); + row.setStatus(status); + row.setStatusChangedAt(LocalDateTime.now()); + row.setStatusChangedBy(actor); + FileEncryptionKey saved = repository.save(row); + invalidate(keyId); + return saved; + } + + /** + * Reverses the kill switch. Returns the key to ACTIVE only while its scope has no other ACTIVE + * key, otherwise to RETIRED — which unwraps existing content just the same, and keeps exactly + * one key wrapping new writes per scope. + * + *

The second case is reached whenever the scope uploaded anything while revoked: those + * writes minted a fresh ACTIVE key, since revoking a key blocks reads of existing content + * rather than stopping the scope from storing new files. + */ + public FileEncryptionKey enable(UUID keyId, String actor) throws StorageEncryptionException { + FileEncryptionKey row = + repository + .findById(keyId) + .orElseThrow( + () -> new StorageEncryptionException("No encryption key " + keyId)); + boolean scopeHasAnotherActiveKey = + activeForScope(row.getScopeType(), row.getScopeId()) + .filter(other -> !other.getKeyId().equals(keyId)) + .isPresent(); + FileEncryptionKey.Status target = + scopeHasAnotherActiveKey + ? FileEncryptionKey.Status.RETIRED + : FileEncryptionKey.Status.ACTIVE; + if (scopeHasAnotherActiveKey) { + log.info( + "Enabling key {} as RETIRED: {}:{} already has an active key wrapping new" + + " writes. Existing content under {} is readable again.", + keyId, + row.getScopeType(), + row.getScopeId(), + keyId); + } + return setKeyStatus(keyId, target, actor); + } + + /** + * Drops cached material for a key so status changes take effect without waiting out the TTL. + */ + public void invalidate(UUID keyId) { + unwrapCache.invalidate(keyId); + activeScopeCache.asMap().values().removeIf(keyId::equals); + } + + /** + * Re-wraps every KEK row below the configured master-key version under the primary master key. + * Cheap by design: touches only this small table, never file contents. Returns the number of + * rows re-wrapped. + */ + public int rotateMasterKey() throws StorageEncryptionException { + int rewrapped = 0; + for (FileEncryptionKey row : + repository.findByMasterKeyVersionLessThan(masterKey.currentVersion())) { + byte[] kek = unwrapRow(row); + row.setWrappedKey( + Base64.getEncoder() + .encodeToString(masterKey.wrap(kek, aadFor(row.getKeyId())))); + row.setMasterKeyVersion(masterKey.currentVersion()); + repository.save(row); + invalidate(row.getKeyId()); + rewrapped++; + } + return rewrapped; + } + + /** + * Startup self-check: proves the configured master key can unwrap every KEK row, so a + * wrong or half-rotated key fails fast instead of silently writing new files under a second key + * hierarchy — or leaving some scopes' files unreadable while the rest of the app looks healthy. + * Checking all rows rather than a sample matters because rotation can stop part-way; the table + * holds one row per scope per rotation, so this stays cheap. */ public void verifyMasterKey() { - repository - .findFirstByStatus(FileEncryptionKey.Status.ACTIVE) - .or(() -> repository.findFirstByStatus(FileEncryptionKey.Status.RETIRED)) - .ifPresent( - row -> { - try { - unwrapRow(row); - } catch (StorageEncryptionException e) { - throw new IllegalStateException( - "The configured file encryption key (fingerprint " - + masterKey.fingerprint() - + ") cannot unwrap existing key " - + row.getKeyId() - + ". Refusing to start with a mismatched key —" - + " restore the original" - + " STIRLING_FILE_ENCRYPTION_KEY /" - + " file-encryption.key.", - e); - } - }); + long pending = repository.countByMasterKeyVersionLessThan(masterKey.currentVersion()); + if (pending > 0) { + log.warn( + "{} encryption key row(s) are still wrapped by the previous master key. Run" + + " POST /api/v1/admin/storage-encryption/master/rotate, then remove" + + " stirling.security.fileEncryptionKeyPrevious.", + pending); + } + long stranded = repository.countByMasterKeyVersionGreaterThan(masterKey.currentVersion()); + if (stranded > 0) { + log.warn( + "{} encryption key row(s) are wrapped by a master-key version newer than the" + + " configured stirling.security.fileEncryptionKeyVersion={}. Rotation" + + " only re-wraps rows below the configured version, so these rows can" + + " never be re-wrapped while it stays this low.", + stranded, + masterKey.currentVersion()); + } + List unreadable = new ArrayList<>(); + StorageEncryptionException firstFailure = null; + // DISABLED rows are included: revocation is meant to be reversible, and a row that cannot + // be unwrapped would not come back on enable. + for (FileEncryptionKey row : repository.findAll()) { + try { + unwrapRow(row); + } catch (StorageEncryptionException e) { + unreadable.add(row); + if (firstFailure == null) { + firstFailure = e; + } + } + } + if (!unreadable.isEmpty()) { + throw new IllegalStateException( + "The configured file encryption key (fingerprint " + + masterKey.fingerprint() + + ") cannot unwrap " + + unreadable.size() + + " of " + + repository.count() + + " encryption key row(s), starting with " + + unreadable.get(0).getKeyId() + + " (" + + unreadable.get(0).getScopeType() + + ":" + + unreadable.get(0).getScopeId() + + ", master key version " + + unreadable.get(0).getMasterKeyVersion() + + "). Files under those keys would be unreadable. Refusing to start —" + + " restore the original key as" + + " stirling.security.fileEncryptionKeyPrevious (or as" + + " STIRLING_FILE_ENCRYPTION_KEY) and re-run the rotation.", + firstFailure); + } } private FileEncryptionKey findOrCreateActive( FileEncryptionKey.ScopeType scopeType, long scopeId) throws StorageEncryptionException { - return repository - .findFirstByScopeTypeAndScopeIdAndStatus( - scopeType, scopeId, FileEncryptionKey.Status.ACTIVE) - .orElseGet(() -> createActive(scopeType, scopeId)); + return activeForScope(scopeType, scopeId).orElseGet(() -> createActive(scopeType, scopeId)); } - // Package-private so the @DataJpaTest can drive the duplicate-insert recovery - // deterministically. + private Optional activeForScope( + FileEncryptionKey.ScopeType scopeType, long scopeId) { + return repository.findFirstByScopeTypeAndScopeIdAndStatusOrderByKeyVersionDesc( + scopeType, scopeId, FileEncryptionKey.Status.ACTIVE); + } + + // Package-private so the @DataJpaTest can drive duplicate-insert recovery deterministically. FileEncryptionKey createActive(FileEncryptionKey.ScopeType scopeType, long scopeId) { byte[] kek = new byte[EncryptedFileFormat.DEK_LENGTH_BYTES]; RANDOM.nextBytes(kek); @@ -168,7 +322,7 @@ public class FileEncryptionKeyService { row.setKeyVersion(version); row.setWrappedKey( Base64.getEncoder().encodeToString(masterKey.wrap(kek, aadFor(row.getKeyId())))); - row.setMasterKeyVersion(FileEncryptionMasterKey.CURRENT_VERSION); + row.setMasterKeyVersion(masterKey.currentVersion()); row.setStatus(FileEncryptionKey.Status.ACTIVE); try { // saveAndFlush inside a fresh transaction so a unique-constraint violation surfaces @@ -180,13 +334,11 @@ public class FileEncryptionKeyService { scopeType, scopeId); unwrapCache.put(saved.getKeyId(), kek); + auditListener.keyCreated(saved.getKeyId(), scopeType + ":" + scopeId, version); return saved; } catch (DataIntegrityViolationException raced) { // Another node created the scope key concurrently; use theirs. - return repository - .findFirstByScopeTypeAndScopeIdAndStatus( - scopeType, scopeId, FileEncryptionKey.Status.ACTIVE) - .orElseThrow(() -> raced); + return activeForScope(scopeType, scopeId).orElseThrow(() -> raced); } } diff --git a/app/proprietary/src/main/java/stirling/software/proprietary/storage/crypto/FileEncryptionMasterKey.java b/app/proprietary/src/main/java/stirling/software/proprietary/storage/crypto/FileEncryptionMasterKey.java index 025b281b79..5b78b00135 100644 --- a/app/proprietary/src/main/java/stirling/software/proprietary/storage/crypto/FileEncryptionMasterKey.java +++ b/app/proprietary/src/main/java/stirling/software/proprietary/storage/crypto/FileEncryptionMasterKey.java @@ -43,16 +43,55 @@ public class FileEncryptionMasterKey { private static final String KEY_FILE = "file-encryption.key"; private static final SecureRandom RANDOM = new SecureRandom(); - /** Bumped when master rotation ships (P2); recorded on every wrapped KEK row. */ + /** Default master-key version when rotation has never been configured. */ public static final int CURRENT_VERSION = 1; private final SecretKey key; + private final SecretKey previousKey; + private final int currentVersion; public FileEncryptionMasterKey(String configuredKey, boolean clusterEnabled) { + this(configuredKey, null, CURRENT_VERSION, clusterEnabled); + } + + /** + * @param previousKeyBase64 optional outgoing master key kept only during rotation: {@link + * #unwrap} falls back to it so existing KEK rows stay readable until {@code rotate} + * re-wraps them under the primary key. + * @param currentVersion admin-bumped version stamped on newly wrapped KEK rows ({@code + * stirling.security.fileEncryptionKeyVersion}); rotation re-wraps rows below it. + */ + public FileEncryptionMasterKey( + String configuredKey, + String previousKeyBase64, + int currentVersion, + boolean clusterEnabled) { this.key = resolveKey(configuredKey, clusterEnabled); + this.previousKey = + previousKeyBase64 == null || previousKeyBase64.isBlank() + ? null + : decodeKey( + previousKeyBase64, "stirling.security.fileEncryptionKeyPrevious"); + if (currentVersion < 1) { + log.warn( + "stirling.security.fileEncryptionKeyVersion={} is not a valid version; using 1", + currentVersion); + } + this.currentVersion = Math.max(1, currentVersion); log.info( - "Storage encryption master key initialised (AES-256-GCM, fingerprint {})", - fingerprint()); + "Storage encryption master key initialised (AES-256-GCM, fingerprint {}, version" + + " {}{})", + fingerprint(), + this.currentVersion, + previousKey != null ? ", previous key configured for rotation" : ""); + } + + public int currentVersion() { + return currentVersion; + } + + public boolean hasPreviousKey() { + return previousKey != null; } private static SecretKey resolveKey(String configuredKey, boolean clusterEnabled) { @@ -152,10 +191,29 @@ public class FileEncryptionMasterKey { } public byte[] unwrap(byte[] wrapped, byte[] associatedData) throws GeneralSecurityException { + try { + return unwrapWith(key, wrapped, associatedData); + } catch (GeneralSecurityException primaryFailure) { + if (previousKey == null) { + throw primaryFailure; + } + try { + return unwrapWith(previousKey, wrapped, associatedData); + } catch (GeneralSecurityException previousFailure) { + // Report the primary key's failure, or a corrupt row gets diagnosed through the + // outgoing key's error message. + primaryFailure.addSuppressed(previousFailure); + throw primaryFailure; + } + } + } + + private static byte[] unwrapWith(SecretKey unwrapKey, byte[] wrapped, byte[] associatedData) + throws GeneralSecurityException { byte[] iv = Arrays.copyOfRange(wrapped, 0, IV_BYTES); byte[] ciphertext = Arrays.copyOfRange(wrapped, IV_BYTES, wrapped.length); Cipher cipher = Cipher.getInstance(TRANSFORMATION); - cipher.init(Cipher.DECRYPT_MODE, key, new GCMParameterSpec(GCM_TAG_BITS, iv)); + cipher.init(Cipher.DECRYPT_MODE, unwrapKey, new GCMParameterSpec(GCM_TAG_BITS, iv)); cipher.updateAAD(associatedData); return cipher.doFinal(ciphertext); } diff --git a/app/proprietary/src/main/java/stirling/software/proprietary/storage/crypto/ResourceUpload.java b/app/proprietary/src/main/java/stirling/software/proprietary/storage/crypto/ResourceUpload.java new file mode 100644 index 0000000000..fa48d3bb39 --- /dev/null +++ b/app/proprietary/src/main/java/stirling/software/proprietary/storage/crypto/ResourceUpload.java @@ -0,0 +1,74 @@ +package stirling.software.proprietary.storage.crypto; + +import java.io.File; +import java.io.IOException; +import java.io.InputStream; +import java.nio.file.Files; + +import org.springframework.core.io.Resource; +import org.springframework.web.multipart.MultipartFile; + +/** + * Presents a loaded {@link Resource} as a {@link MultipartFile} so existing blobs can be re-stored + * through {@code StorageProvider.store(...)} — the encrypt-existing migration re-uploads plaintext + * blobs through the encrypting decorator this way. The size must be accurate: the decorator records + * it as the plaintext length in the blob header. + */ +public class ResourceUpload implements MultipartFile { + + private final Resource resource; + private final String filename; + private final String contentType; + private final long sizeBytes; + + public ResourceUpload(Resource resource, String filename, String contentType, long sizeBytes) { + this.resource = resource; + this.filename = filename; + this.contentType = contentType; + this.sizeBytes = sizeBytes; + } + + @Override + public String getName() { + return "file"; + } + + @Override + public String getOriginalFilename() { + return filename; + } + + @Override + public String getContentType() { + return contentType; + } + + @Override + public boolean isEmpty() { + return sizeBytes == 0; + } + + @Override + public long getSize() { + return sizeBytes; + } + + @Override + public byte[] getBytes() throws IOException { + try (InputStream in = resource.getInputStream()) { + return in.readAllBytes(); + } + } + + @Override + public InputStream getInputStream() throws IOException { + return resource.getInputStream(); + } + + @Override + public void transferTo(File dest) throws IOException { + try (InputStream in = resource.getInputStream()) { + Files.copy(in, dest.toPath()); + } + } +} diff --git a/app/proprietary/src/main/java/stirling/software/proprietary/storage/crypto/StorageEncryptionAuditListener.java b/app/proprietary/src/main/java/stirling/software/proprietary/storage/crypto/StorageEncryptionAuditListener.java new file mode 100644 index 0000000000..978bf3af0d --- /dev/null +++ b/app/proprietary/src/main/java/stirling/software/proprietary/storage/crypto/StorageEncryptionAuditListener.java @@ -0,0 +1,21 @@ +package stirling.software.proprietary.storage.crypto; + +import java.util.UUID; + +/** + * Security-event hook for the storage-encryption layer. The crypto classes stay plain (non-Spring) + * objects, so audit emission is injected through this interface; production wires {@link + * AuditingStorageEncryptionListener}, tests default to {@link #NOOP}. + */ +public interface StorageEncryptionAuditListener { + + StorageEncryptionAuditListener NOOP = new StorageEncryptionAuditListener() {}; + + default void encrypted(String storageKey, UUID keyId) {} + + default void decrypted(String storageKey, UUID keyId) {} + + default void decryptDenied(UUID keyId, String reason) {} + + default void keyCreated(UUID keyId, String scope, int version) {} +} diff --git a/app/proprietary/src/main/java/stirling/software/proprietary/storage/crypto/StorageEncryptionErrors.java b/app/proprietary/src/main/java/stirling/software/proprietary/storage/crypto/StorageEncryptionErrors.java new file mode 100644 index 0000000000..f04e5a5dcc --- /dev/null +++ b/app/proprietary/src/main/java/stirling/software/proprietary/storage/crypto/StorageEncryptionErrors.java @@ -0,0 +1,22 @@ +package stirling.software.proprietary.storage.crypto; + +import org.springframework.http.HttpStatus; +import org.springframework.web.server.ResponseStatusException; + +/** + * Shared HTTP translation for encryption failures, so every path that serves stored bytes answers a + * revoked key the same way. {@link StorageKeyRevokedException} extends {@code IOException}, so a + * caller that only catches {@code IOException} reports a deliberate, reversible policy state as a + * server fault. + */ +public final class StorageEncryptionErrors { + + private StorageEncryptionErrors() {} + + public static ResponseStatusException revoked(StorageKeyRevokedException cause) { + return new ResponseStatusException( + HttpStatus.FORBIDDEN, + "Access to this file has been revoked (its encryption key is disabled)", + cause); + } +} diff --git a/app/proprietary/src/main/java/stirling/software/proprietary/storage/crypto/StorageEncryptionState.java b/app/proprietary/src/main/java/stirling/software/proprietary/storage/crypto/StorageEncryptionState.java index 8fe465406b..0d2bd112fc 100644 --- a/app/proprietary/src/main/java/stirling/software/proprietary/storage/crypto/StorageEncryptionState.java +++ b/app/proprietary/src/main/java/stirling/software/proprietary/storage/crypto/StorageEncryptionState.java @@ -10,12 +10,13 @@ import stirling.software.proprietary.storage.repository.FileEncryptionKeyReposit /** * Holds the storage-encryption machinery for the always-installed {@link EncryptingStorageProvider} - * decorator. + * decorator, shared with the admin API and migration job so kill-switch cache invalidation hits the + * caches the decorator reads. * *

The decorator is unconditional so a node whose config lags the cluster (flag off, rolling * deploy, config drift) can never stream raw ciphertext: it always sniffs the header and decrypts - * or fails loudly. The expensive parts — resolving the master key (which may generate a key file) - * and the key service — are created lazily: eagerly at startup only when the write flag is on or + * or fails loudly. The expensive parts - resolving the master key (which may generate a key file) + * and the key service - are created lazily: eagerly at startup only when the write flag is on or * key rows already exist (preserving the fail-fast master-key verification), otherwise on first * encounter with an encrypted blob. */ @@ -27,6 +28,7 @@ public class StorageEncryptionState { private final boolean writeEnabled; private final Supplier keyServiceFactory; private final FileEncryptionKeyRepository keyRepository; + private final StorageEncryptionAuditListener auditListener; private volatile FileEncryptionKeyService keyService; private volatile boolean keysExistEverChecked; @@ -36,17 +38,27 @@ public class StorageEncryptionState { public StorageEncryptionState( boolean writeEnabled, Supplier keyServiceFactory, - FileEncryptionKeyRepository keyRepository) { + FileEncryptionKeyRepository keyRepository, + StorageEncryptionAuditListener auditListener) { this.writeEnabled = writeEnabled; this.keyServiceFactory = keyServiceFactory; this.keyRepository = keyRepository; + this.auditListener = auditListener; } /** Test convenience: a pre-materialised state around an existing service. */ public static StorageEncryptionState of( boolean writeEnabled, FileEncryptionKeyService keyService) { + return of(writeEnabled, keyService, StorageEncryptionAuditListener.NOOP); + } + + /** Test convenience: a pre-materialised state with an explicit audit listener. */ + public static StorageEncryptionState of( + boolean writeEnabled, + FileEncryptionKeyService keyService, + StorageEncryptionAuditListener auditListener) { StorageEncryptionState state = - new StorageEncryptionState(writeEnabled, () -> keyService, null); + new StorageEncryptionState(writeEnabled, () -> keyService, null, auditListener); state.keyService = keyService; return state; } @@ -56,9 +68,18 @@ public class StorageEncryptionState { return writeEnabled; } + /** True once the key machinery has been materialised (eagerly at boot or on first use). */ + public boolean isMaterialised() { + return keyService != null; + } + + public StorageEncryptionAuditListener auditListener() { + return auditListener; + } + /** * The key service, created on first use. A failure here (no key material, wrong key) is a loud, - * actionable error — never silently-served ciphertext. + * actionable error - never silently-served ciphertext. */ public FileEncryptionKeyService keyService() throws StorageEncryptionException { FileEncryptionKeyService current = keyService; diff --git a/app/proprietary/src/main/java/stirling/software/proprietary/storage/model/api/StorageEncryptionStatusResponse.java b/app/proprietary/src/main/java/stirling/software/proprietary/storage/model/api/StorageEncryptionStatusResponse.java new file mode 100644 index 0000000000..0699a1aecd --- /dev/null +++ b/app/proprietary/src/main/java/stirling/software/proprietary/storage/model/api/StorageEncryptionStatusResponse.java @@ -0,0 +1,30 @@ +package stirling.software.proprietary.storage.model.api; + +import java.time.LocalDateTime; +import java.util.List; +import java.util.UUID; + +/** + * Admin view of storage encryption at rest. The master-key fingerprint (SHA-256 prefix, never key + * material) lets admins verify their key backup matches the live key. + */ +public record StorageEncryptionStatusResponse( + boolean writeEnabled, + boolean active, + String masterKeyFingerprint, + Integer masterKeyVersion, + long encryptedFiles, + long plaintextFiles, + List keys) { + + public record KeyInfo( + UUID keyId, + String scopeType, + long scopeId, + int keyVersion, + int masterKeyVersion, + String status, + LocalDateTime createdAt, + LocalDateTime statusChangedAt, + String statusChangedBy) {} +} diff --git a/app/proprietary/src/main/java/stirling/software/proprietary/storage/repository/FileEncryptionKeyRepository.java b/app/proprietary/src/main/java/stirling/software/proprietary/storage/repository/FileEncryptionKeyRepository.java index b6563f8c26..90450ee4ad 100644 --- a/app/proprietary/src/main/java/stirling/software/proprietary/storage/repository/FileEncryptionKeyRepository.java +++ b/app/proprietary/src/main/java/stirling/software/proprietary/storage/repository/FileEncryptionKeyRepository.java @@ -1,5 +1,6 @@ package stirling.software.proprietary.storage.repository; +import java.util.List; import java.util.Optional; import java.util.UUID; @@ -11,11 +12,29 @@ import stirling.software.proprietary.storage.model.FileEncryptionKey; @Repository public interface FileEncryptionKeyRepository extends JpaRepository { - Optional findFirstByScopeTypeAndScopeIdAndStatus( + /** + * Ordered so that if a scope ever holds more than one row in the given status, every node picks + * the same one instead of following database row order. + */ + Optional findFirstByScopeTypeAndScopeIdAndStatusOrderByKeyVersionDesc( FileEncryptionKey.ScopeType scopeType, long scopeId, FileEncryptionKey.Status status); Optional findFirstByScopeTypeAndScopeIdOrderByKeyVersionDesc( FileEncryptionKey.ScopeType scopeType, long scopeId); Optional findFirstByStatus(FileEncryptionKey.Status status); + + /** + * Rows still wrapped by an older master key. Counting and fetching are separate so the startup + * check can ask the database for a number instead of materialising every row. + */ + long countByMasterKeyVersionLessThan(int masterKeyVersion); + + List findByMasterKeyVersionLessThan(int masterKeyVersion); + + /** + * Rows wrapped by a version the configuration has since gone below. Rotation only re-wraps rows + * under the configured version, so these can never be re-wrapped. + */ + long countByMasterKeyVersionGreaterThan(int masterKeyVersion); } diff --git a/app/proprietary/src/main/java/stirling/software/proprietary/storage/repository/StoredFileRepository.java b/app/proprietary/src/main/java/stirling/software/proprietary/storage/repository/StoredFileRepository.java index 47545ab58a..b7ff77c987 100644 --- a/app/proprietary/src/main/java/stirling/software/proprietary/storage/repository/StoredFileRepository.java +++ b/app/proprietary/src/main/java/stirling/software/proprietary/storage/repository/StoredFileRepository.java @@ -3,6 +3,7 @@ package stirling.software.proprietary.storage.repository; import java.util.List; import java.util.Optional; +import org.springframework.data.domain.Pageable; import org.springframework.data.jpa.repository.JpaRepository; import org.springframework.data.jpa.repository.Modifying; import org.springframework.data.jpa.repository.Query; @@ -73,4 +74,53 @@ public interface StoredFileRepository extends JpaRepository { + "WHERE sf.workflowSession IN " + "(SELECT ws FROM WorkflowSession ws WHERE ws.owner = :user)") void clearWorkflowSessionReferencesByOwner(@Param("user") User user); + + // ---- storage encryption at rest ---------------------------------------------------- + + long countByEncryptionKeyIdIsNull(); + + long countByEncryptionKeyIdIsNotNull(); + + /** + * Next batch of plaintext files for the encrypt-existing migration. Cursor-based ({@code id > + * lastId}) so per-file failures don't wedge the loop, and owner is fetched eagerly because the + * job re-stores blobs under the owner's scope key outside a web transaction. + */ + @Query( + "SELECT f FROM StoredFile f JOIN FETCH f.owner " + + "WHERE f.encryptionKeyId IS NULL AND f.id > :lastId ORDER BY f.id ASC") + List findMigratableAfter(@Param("lastId") long lastId, Pageable pageable); + + /** + * Compare-and-swap updates for the migration: each blob's storage key only flips if it still + * holds the value the job read, so a user replacing the file mid-migration wins and the job + * discards its own copy. The main-blob swap also stamps the key id, which is what removes the + * row from the migration's selection. + */ + @Modifying + @Transactional + @Query( + "UPDATE StoredFile f SET f.storageKey = :newKey, f.encryptionKeyId = :keyId " + + "WHERE f.id = :id AND f.storageKey = :oldKey") + int swapMainBlob( + @Param("id") Long id, + @Param("oldKey") String oldKey, + @Param("newKey") String newKey, + @Param("keyId") String keyId); + + @Modifying + @Transactional + @Query( + "UPDATE StoredFile f SET f.historyStorageKey = :newKey " + + "WHERE f.id = :id AND f.historyStorageKey = :oldKey") + int swapHistoryBlob( + @Param("id") Long id, @Param("oldKey") String oldKey, @Param("newKey") String newKey); + + @Modifying + @Transactional + @Query( + "UPDATE StoredFile f SET f.auditLogStorageKey = :newKey " + + "WHERE f.id = :id AND f.auditLogStorageKey = :oldKey") + int swapAuditLogBlob( + @Param("id") Long id, @Param("oldKey") String oldKey, @Param("newKey") String newKey); } diff --git a/app/proprietary/src/main/java/stirling/software/proprietary/storage/service/FileStorageService.java b/app/proprietary/src/main/java/stirling/software/proprietary/storage/service/FileStorageService.java index 9764f392b1..7b7d9b68a6 100644 --- a/app/proprietary/src/main/java/stirling/software/proprietary/storage/service/FileStorageService.java +++ b/app/proprietary/src/main/java/stirling/software/proprietary/storage/service/FileStorageService.java @@ -32,6 +32,7 @@ import stirling.software.common.model.ApplicationProperties; import stirling.software.proprietary.security.database.repository.UserRepository; import stirling.software.proprietary.security.model.User; import stirling.software.proprietary.security.service.EmailService; +import stirling.software.proprietary.storage.crypto.StorageEncryptionErrors; import stirling.software.proprietary.storage.crypto.StorageKeyRevokedException; import stirling.software.proprietary.storage.model.FileShare; import stirling.software.proprietary.storage.model.FileShareAccess; @@ -496,16 +497,11 @@ public class FileStorageService { try { return storageProvider.load(file.getStorageKey()); } catch (StorageKeyRevokedException e) { - // Deliberate, reversible policy state (encryption key disabled), not a server fault — - // surface as forbidden so the client sees "revoked", not "internal error". log.warn( "Access to stored file {} denied: {}", file != null ? file.getId() : null, e.getMessage()); - throw new ResponseStatusException( - HttpStatus.FORBIDDEN, - "Access to this file has been revoked (its encryption key is disabled)", - e); + throw StorageEncryptionErrors.revoked(e); } catch (IOException e) { log.error( "Failed to load stored file {} (key: {})", diff --git a/app/proprietary/src/main/java/stirling/software/proprietary/storage/service/StorageEncryptionMigrationService.java b/app/proprietary/src/main/java/stirling/software/proprietary/storage/service/StorageEncryptionMigrationService.java new file mode 100644 index 0000000000..6df4d1aaea --- /dev/null +++ b/app/proprietary/src/main/java/stirling/software/proprietary/storage/service/StorageEncryptionMigrationService.java @@ -0,0 +1,316 @@ +package stirling.software.proprietary.storage.service; + +import java.io.IOException; +import java.time.Instant; +import java.util.List; +import java.util.Map; +import java.util.Optional; +import java.util.concurrent.atomic.AtomicLong; +import java.util.concurrent.atomic.AtomicReference; + +import org.springframework.core.io.Resource; +import org.springframework.data.domain.PageRequest; +import org.springframework.stereotype.Service; + +import lombok.extern.slf4j.Slf4j; + +import stirling.software.proprietary.audit.AuditEventType; +import stirling.software.proprietary.security.model.User; +import stirling.software.proprietary.service.AuditService; +import stirling.software.proprietary.storage.crypto.ResourceUpload; +import stirling.software.proprietary.storage.crypto.StorageEncryptionState; +import stirling.software.proprietary.storage.model.StoredFile; +import stirling.software.proprietary.storage.provider.StorageProvider; +import stirling.software.proprietary.storage.provider.StoredObject; +import stirling.software.proprietary.storage.repository.StoredFileRepository; + +/** + * Encrypts the pre-existing plaintext backlog after storage encryption is enabled (new writes are + * encrypted from the moment the flag is on; this job converts what was stored before). + * + *

Crash-safe per file: store the encrypted copy under a NEW storage key, compare-and-swap the + * row, and only then delete the old blob. A CAS miss means a user replaced the file mid-migration — + * the job discards its own copy and moves on. The worst crash outcome is an orphaned new blob, + * never a lost file, and re-runs are idempotent because selection is {@code encryptionKeyId IS + * NULL} (only stamped by the final main-blob swap). + */ +@Service +@Slf4j +public class StorageEncryptionMigrationService { + + private static final int PAGE_SIZE = 25; + private static final long PAUSE_BETWEEN_PAGES_MS = 200; + + private final StoredFileRepository storedFileRepository; + private final StorageProvider storageProvider; + private final StorageEncryptionState encryptionState; + private final AuditService auditService; + + private final AtomicReference currentRun = new AtomicReference<>(); + + public StorageEncryptionMigrationService( + StoredFileRepository storedFileRepository, + StorageProvider storageProvider, + StorageEncryptionState encryptionState, + AuditService auditService) { + this.storedFileRepository = storedFileRepository; + this.storageProvider = storageProvider; + this.encryptionState = encryptionState; + this.auditService = auditService; + } + + public enum State { + RUNNING, + COMPLETED, + FAILED + } + + public record MigrationStatus( + State state, + long total, + long processed, + long skipped, + long failed, + Instant startedAt, + Instant finishedAt) {} + + /** Starts the migration; throws {@link IllegalStateException} if one is already running. */ + public MigrationStatus start() { + if (!encryptionState.isWriteEnabled()) { + throw new IllegalStateException( + "storage.encryption.enabled must be on before migrating existing files"); + } + // Captured here because the run itself executes on a virtual thread with no security + // context: without this, re-encrypting every stored file is attributed to "system". + String principal = auditService.captureCurrentPrincipal(); + Run run = new Run(storedFileRepository.countByEncryptionKeyIdIsNull(), principal); + Run previous = currentRun.get(); + if (previous != null && previous.state == State.RUNNING) { + throw new IllegalStateException("A migration is already running"); + } + if (!currentRun.compareAndSet(previous, run)) { + throw new IllegalStateException("A migration is already running"); + } + auditService.audit( + principal, + AuditEventType.STORAGE_ENCRYPTION, + Map.of("action", "migration.started", "plaintextFiles", run.total)); + Thread.ofVirtual().name("storage-encryption-migration").start(() -> execute(run)); + return run.snapshot(); + } + + /** Latest run's progress, or empty if none has started this uptime. */ + public Optional status() { + Run run = currentRun.get(); + return Optional.ofNullable(run).map(Run::snapshot); + } + + private void execute(Run run) { + log.info("Storage encryption migration started ({} plaintext files)", run.total); + State terminal; + try { + terminal = migratePages(run); + } catch (InterruptedException e) { + Thread.currentThread().interrupt(); + log.warn("Storage encryption migration interrupted"); + terminal = State.FAILED; + } catch (Exception e) { + log.error("Storage encryption migration aborted", e); + terminal = State.FAILED; + } + run.finish(terminal); + MigrationStatus done = run.snapshot(); + log.info( + "Storage encryption migration finished: {} encrypted, {} skipped, {} failed", + done.processed(), + done.skipped(), + done.failed()); + auditService.audit( + run.principal, + AuditEventType.STORAGE_ENCRYPTION, + Map.of( + "action", "migration.completed", + "state", done.state().name(), + "encrypted", done.processed(), + "skipped", done.skipped(), + "failed", done.failed())); + } + + /** Walks the plaintext backlog a page at a time; returns the state the run ends in. */ + private State migratePages(Run run) throws InterruptedException { + long lastId = 0; + while (true) { + List page = + storedFileRepository.findMigratableAfter(lastId, PageRequest.of(0, PAGE_SIZE)); + if (page.isEmpty()) { + return State.COMPLETED; + } + for (StoredFile file : page) { + if (!encryptionState.isWriteEnabled()) { + // Run-level condition, not a per-file failure: carrying on would copy every + // remaining file as plaintext and delete it again. + log.warn( + "Encryption write path turned off mid-run; stopping with {} of {} files" + + " encrypted. Re-enable storage.encryption.enabled and start" + + " the migration again.", + run.processed.get(), + run.total); + return State.FAILED; + } + lastId = file.getId(); + try { + migrateFile(file, run); + } catch (Exception e) { + run.failed.incrementAndGet(); + log.error( + "Failed to encrypt stored file {} (key {})", + file.getId(), + file.getStorageKey(), + e); + } + } + Thread.sleep(PAUSE_BETWEEN_PAGES_MS); + } + } + + /** + * The three blobs are swapped independently, so a compare-and-swap miss on a secondary one + * abandons the whole file for this run: the row keeps {@code encryptionKeyId} null and the next + * run picks it up from the top. + */ + private void migrateFile(StoredFile file, Run run) throws IOException { + User owner = file.getOwner(); + + // Secondary blobs first; the main-blob swap stamps encryptionKeyId and thereby removes + // the row from the migration's selection, so it must come last. + if (file.getHistoryStorageKey() != null) { + String oldKey = file.getHistoryStorageKey(); + String newKey = + reencrypt( + owner, + oldKey, + file.getHistoryFilename(), + file.getHistoryContentType(), + file.getHistorySizeBytes()) + .getStorageKey(); + if (storedFileRepository.swapHistoryBlob(file.getId(), oldKey, newKey) == 1) { + deleteQuietly(oldKey); + } else { + deleteQuietly(newKey); + run.skipped.incrementAndGet(); + return; + } + } + if (file.getAuditLogStorageKey() != null) { + String oldKey = file.getAuditLogStorageKey(); + String newKey = + reencrypt( + owner, + oldKey, + file.getAuditLogFilename(), + file.getAuditLogContentType(), + file.getAuditLogSizeBytes()) + .getStorageKey(); + if (storedFileRepository.swapAuditLogBlob(file.getId(), oldKey, newKey) == 1) { + deleteQuietly(oldKey); + } else { + deleteQuietly(newKey); + run.skipped.incrementAndGet(); + return; + } + } + + String oldKey = file.getStorageKey(); + StoredObject encrypted = + reencrypt( + owner, + oldKey, + file.getOriginalFilename(), + file.getContentType(), + file.getSizeBytes()); + if (encrypted.getEncryptionKeyId() == null) { + // The flag flipped between this page's check and the store; the loop stops on the next + // file, so this only ever discards one plaintext copy. + deleteQuietly(encrypted.getStorageKey()); + throw new IllegalStateException("Encryption write path is no longer active"); + } + if (storedFileRepository.swapMainBlob( + file.getId(), + oldKey, + encrypted.getStorageKey(), + encrypted.getEncryptionKeyId()) + == 1) { + deleteQuietly(oldKey); + run.processed.incrementAndGet(); + } else { + deleteQuietly(encrypted.getStorageKey()); + run.skipped.incrementAndGet(); + } + } + + private StoredObject reencrypt( + User owner, String storageKey, String filename, String contentType, Long sizeBytes) + throws IOException { + Resource plaintext = storageProvider.load(storageKey); + long size = -1; + try { + size = plaintext.contentLength(); + } catch (IOException | RuntimeException e) { + // Some resources refuse contentLength() once partially read. + size = -1; + } + if (size < 0) { + // A resource may also simply report -1; the row's recorded plaintext size is + // authoritative, so prefer it over failing the file. + size = sizeBytes != null ? sizeBytes : -1; + } + if (size < 0) { + throw new IOException("Cannot determine plaintext size for " + storageKey); + } + return storageProvider.store( + owner, new ResourceUpload(plaintext, filename, contentType, size)); + } + + private void deleteQuietly(String storageKey) { + try { + storageProvider.delete(storageKey); + } catch (IOException e) { + log.warn("Could not delete blob {} after migration step; orphaned", storageKey, e); + } + } + + private static final class Run { + private final long total; + + /** The admin who started the run, so the completion event is attributed to them. */ + private final String principal; + + private final Instant startedAt = Instant.now(); + private final AtomicLong processed = new AtomicLong(); + private final AtomicLong skipped = new AtomicLong(); + private final AtomicLong failed = new AtomicLong(); + private volatile State state = State.RUNNING; + private volatile Instant finishedAt; + + private Run(long total, String principal) { + this.total = total; + this.principal = principal; + } + + private void finish(State terminal) { + this.state = terminal; + this.finishedAt = Instant.now(); + } + + private MigrationStatus snapshot() { + return new MigrationStatus( + state, + total, + processed.get(), + skipped.get(), + failed.get(), + startedAt, + finishedAt); + } + } +} diff --git a/app/proprietary/src/main/java/stirling/software/proprietary/workflow/service/WorkflowSessionService.java b/app/proprietary/src/main/java/stirling/software/proprietary/workflow/service/WorkflowSessionService.java index 41aed7d9c2..4c60c60df2 100644 --- a/app/proprietary/src/main/java/stirling/software/proprietary/workflow/service/WorkflowSessionService.java +++ b/app/proprietary/src/main/java/stirling/software/proprietary/workflow/service/WorkflowSessionService.java @@ -37,6 +37,8 @@ import lombok.extern.slf4j.Slf4j; import stirling.software.common.model.ApplicationProperties; import stirling.software.proprietary.security.database.repository.UserRepository; import stirling.software.proprietary.security.model.User; +import stirling.software.proprietary.storage.crypto.StorageEncryptionErrors; +import stirling.software.proprietary.storage.crypto.StorageKeyRevokedException; import stirling.software.proprietary.storage.model.FilePurpose; import stirling.software.proprietary.storage.model.ShareAccessRole; import stirling.software.proprietary.storage.model.StoredFile; @@ -444,9 +446,21 @@ public class WorkflowSessionService { HttpStatus.NOT_FOUND, "No processed file available for this session"); } - String storageKey = session.getProcessedFile().getStorageKey(); - org.springframework.core.io.Resource resource = storageProvider.load(storageKey); - return resource.getContentAsByteArray(); + return readBlob(session.getProcessedFile().getStorageKey()); + } + + /** + * Reads a stored blob, translating a revoked encryption key into 403 the same way {@code + * FileStorageService} does. Without this the callers' {@code IOException} handling would report + * an administrator's deliberate revocation as a server error. + */ + private byte[] readBlob(String storageKey) throws IOException { + try { + return storageProvider.load(storageKey).getContentAsByteArray(); + } catch (StorageKeyRevokedException e) { + log.warn("Access to workflow blob {} denied: {}", storageKey, e.getMessage()); + throw StorageEncryptionErrors.revoked(e); + } } /** Retrieves the original file data for a workflow session. */ @@ -458,9 +472,7 @@ public class WorkflowSessionService { HttpStatus.NOT_FOUND, "Original file no longer available (session may be finalized)"); } - String storageKey = session.getOriginalFile().getStorageKey(); - org.springframework.core.io.Resource resource = storageProvider.load(storageKey); - return resource.getContentAsByteArray(); + return readBlob(session.getOriginalFile().getStorageKey()); } /** Deletes a workflow session and associated files. */ @@ -658,9 +670,7 @@ public class WorkflowSessionService { } try { - org.springframework.core.io.Resource resource = - storageProvider.load(fileToServe.getStorageKey()); - return resource.getContentAsByteArray(); + return readBlob(fileToServe.getStorageKey()); } catch (IOException e) { log.error("Failed to retrieve document for session {}", sessionId, e); throw new ResponseStatusException( diff --git a/app/proprietary/src/test/java/stirling/software/proprietary/storage/config/StorageProviderConfigTest.java b/app/proprietary/src/test/java/stirling/software/proprietary/storage/config/StorageProviderConfigTest.java index a6fd31e9b2..4098d6ab19 100644 --- a/app/proprietary/src/test/java/stirling/software/proprietary/storage/config/StorageProviderConfigTest.java +++ b/app/proprietary/src/test/java/stirling/software/proprietary/storage/config/StorageProviderConfigTest.java @@ -24,8 +24,10 @@ import stirling.software.proprietary.model.Team; import stirling.software.proprietary.security.configuration.ee.KeygenLicenseVerifier.License; import stirling.software.proprietary.security.configuration.ee.LicenseKeyChecker; import stirling.software.proprietary.security.model.User; +import stirling.software.proprietary.service.AuditService; import stirling.software.proprietary.storage.crypto.EncryptingStorageProvider; import stirling.software.proprietary.storage.crypto.InMemoryKeyRepo; +import stirling.software.proprietary.storage.crypto.StorageEncryptionAuditListener; import stirling.software.proprietary.storage.crypto.StorageEncryptionState; import stirling.software.proprietary.storage.provider.StorageProvider; import stirling.software.proprietary.storage.repository.FileEncryptionKeyRepository; @@ -44,12 +46,16 @@ class StorageProviderConfigTest { private final InMemoryKeyRepo keyRepo = new InMemoryKeyRepo(); private final PlatformTransactionManager txManager = mock(PlatformTransactionManager.class); + private StorageEncryptionState newState(StorageProviderConfig cfg) { + return cfg.storageEncryptionState(MASTER, "", 1, false, txManager); + } + // ---- decorator installation matrix ------------------------------------------------- @Test void decorator_alwaysInstalled_evenWhenEncryptionOffAndNoKeys() { StorageProviderConfig cfg = newConfig("local", License.NORMAL, false); - StorageEncryptionState state = cfg.storageEncryptionState(MASTER, false, txManager); + StorageEncryptionState state = newState(cfg); StorageProvider provider = cfg.storageProvider(state, Optional.empty()); assertThat(provider).isInstanceOf(EncryptingStorageProvider.class); @@ -61,7 +67,7 @@ class StorageProviderConfigTest { @Test void decorator_writeEnabled_requiresLicenceAndSuppressesDirectDownloads() { StorageProviderConfig cfg = newConfig("local", License.SERVER, true); - StorageEncryptionState state = cfg.storageEncryptionState(MASTER, false, txManager); + StorageEncryptionState state = newState(cfg); StorageProvider provider = cfg.storageProvider(state, Optional.empty()); assertThat(provider).isInstanceOf(EncryptingStorageProvider.class); @@ -73,7 +79,7 @@ class StorageProviderConfigTest { void decorator_flagOffButKeysExist_decryptOnlyModeStillMaterialises() throws Exception { // Drifted node: keys created elsewhere; storage on, as it must be to serve files. StorageProviderConfig seedCfg = newConfig("local", License.SERVER, true, true); - StorageEncryptionState seedState = seedCfg.storageEncryptionState(MASTER, false, txManager); + StorageEncryptionState seedState = newState(seedCfg); Team team = new Team(); team.setId(1L); User owner = new User(); @@ -81,7 +87,7 @@ class StorageProviderConfigTest { seedState.keyService().activeKekForOwner(owner); StorageProviderConfig cfg = newConfig("local", License.NORMAL, false, true); - StorageEncryptionState state = cfg.storageEncryptionState(MASTER, false, txManager); + StorageEncryptionState state = newState(cfg); assertThat(cfg.storageProvider(state, Optional.empty())) .isInstanceOf(EncryptingStorageProvider.class); @@ -89,13 +95,13 @@ class StorageProviderConfigTest { // Encrypted content may exist -> presigned URLs must be suppressed on this node too. assertThat(state.suppressDirectDownloads()).isTrue(); // Eager init ran (keys existed at boot), so decryption works without a licence. - assertThat(state.keyService()).isNotNull(); + assertThat(state.isMaterialised()).isTrue(); } @Test void encryption_enabled_normalLicense_failsStartup() { StorageProviderConfig cfg = newConfig("local", License.NORMAL, true); - assertThatThrownBy(() -> cfg.storageEncryptionState(MASTER, false, txManager)) + assertThatThrownBy(() -> newState(cfg)) .isInstanceOf(IllegalStateException.class) .hasMessageContaining("storage.encryption requires a Pro or Enterprise license"); } @@ -104,7 +110,7 @@ class StorageProviderConfigTest { void encryption_enabled_wrongLengthKey_failsStartup() { StorageProviderConfig cfg = newConfig("local", License.SERVER, true); String shortKey = Base64.getEncoder().encodeToString("only16bytes-yes!".getBytes()); - assertThatThrownBy(() -> cfg.storageEncryptionState(shortKey, false, txManager)) + assertThatThrownBy(() -> cfg.storageEncryptionState(shortKey, "", 1, false, txManager)) .isInstanceOf(IllegalStateException.class) .hasMessageContaining("32 bytes"); } @@ -114,16 +120,18 @@ class StorageProviderConfigTest { @Test void storageDisabled_neverQueriesTheKeyRegistry() { StorageProviderConfig cfg = newConfig("local", License.NORMAL, false); - StorageEncryptionState state = cfg.storageEncryptionState(MASTER, false, txManager); + StorageEncryptionState state = newState(cfg); assertThat(state.isWriteEnabled()).isFalse(); + // No probe means no eager init, so no master key is resolved on a node that never stores. + assertThat(state.isMaterialised()).isFalse(); verify(keyRepo.mock, never()).count(); } @Test void storageDisabled_decoratorStillInstalledSoCiphertextIsNeverServedRaw() { StorageProviderConfig cfg = newConfig("local", License.NORMAL, false); - StorageEncryptionState state = cfg.storageEncryptionState(MASTER, false, txManager); + StorageEncryptionState state = newState(cfg); assertThat(cfg.storageProvider(state, Optional.empty())) .isInstanceOf(EncryptingStorageProvider.class); @@ -134,7 +142,9 @@ class StorageProviderConfigTest { FileEncryptionKeyRepository broken = mock(FileEncryptionKeyRepository.class); when(broken.count()) .thenThrow(new InvalidDataAccessResourceUsageException("no such table")); - StorageEncryptionState state = new StorageEncryptionState(false, () -> null, broken); + StorageEncryptionState state = + new StorageEncryptionState( + false, () -> null, broken, StorageEncryptionAuditListener.NOOP); assertThat(state.encryptedContentMayExist()).isFalse(); assertThat(state.suppressDirectDownloads()).isTrue(); @@ -143,7 +153,7 @@ class StorageProviderConfigTest { @Test void storageEnabledWithoutEncryption_probesRegistry() { StorageProviderConfig cfg = newConfig("local", License.NORMAL, false, true); - cfg.storageEncryptionState(MASTER, false, txManager); + newState(cfg); verify(keyRepo.mock, atLeastOnce()).count(); } @@ -152,7 +162,7 @@ class StorageProviderConfigTest { @Test void provider_s3_normalLicense_throwsBeforeBuildingClient() { StorageProviderConfig cfg = newConfig("s3", License.NORMAL, false); - StorageEncryptionState state = cfg.storageEncryptionState(MASTER, false, txManager); + StorageEncryptionState state = newState(cfg); // License check must throw BEFORE S3Clients.build tries to validate endpoint / bucket. assertThatThrownBy(() -> cfg.storageProvider(state, Optional.empty())) @@ -163,7 +173,7 @@ class StorageProviderConfigTest { @Test void provider_database_normalLicense_throws() { StorageProviderConfig cfg = newConfig("database", License.NORMAL, false); - StorageEncryptionState state = cfg.storageEncryptionState(MASTER, false, txManager); + StorageEncryptionState state = newState(cfg); assertThatThrownBy(() -> cfg.storageProvider(state, Optional.empty())) .isInstanceOf(IllegalStateException.class) @@ -174,7 +184,7 @@ class StorageProviderConfigTest { @Test void provider_database_serverLicense_builds() { StorageProviderConfig cfg = newConfig("database", License.SERVER, false); - StorageEncryptionState state = cfg.storageEncryptionState(MASTER, false, txManager); + StorageEncryptionState state = newState(cfg); assertThatCode(() -> cfg.storageProvider(state, Optional.empty())) .doesNotThrowAnyException(); } @@ -182,7 +192,7 @@ class StorageProviderConfigTest { @Test void provider_s3_serverLicense_passesLicenseCheck_thenFailsOnEmptyConfig() { StorageProviderConfig cfg = newConfig("s3", License.SERVER, false); - StorageEncryptionState state = cfg.storageEncryptionState(MASTER, false, txManager); + StorageEncryptionState state = newState(cfg); // Valid license, but no bucket/endpoint configured - so we expect a CONFIG error, // not a license error. @@ -194,7 +204,7 @@ class StorageProviderConfigTest { @Test void provider_unknown_throwsUnsupportedProvider_notLicense() { StorageProviderConfig cfg = newConfig("magic", License.NORMAL, false); - StorageEncryptionState state = cfg.storageEncryptionState(MASTER, false, txManager); + StorageEncryptionState state = newState(cfg); assertThatThrownBy(() -> cfg.storageProvider(state, Optional.empty())) .isInstanceOf(IllegalStateException.class) @@ -215,6 +225,7 @@ class StorageProviderConfigTest { props.getStorage().getEncryption().setEnabled(encryptionEnabled); StoredFileBlobRepository repo = mock(StoredFileBlobRepository.class); LicenseKeyChecker checker = mock(LicenseKeyChecker.class); + AuditService audit = mock(AuditService.class); when(checker.getPremiumLicenseEnabledResult()).thenReturn(license); if (license == License.SERVER || license == License.ENTERPRISE) { doNothing().when(checker).requireProOrEnterprise(anyString()); @@ -229,6 +240,6 @@ class StorageProviderConfigTest { .when(checker) .requireProOrEnterprise(anyString()); } - return new StorageProviderConfig(props, repo, keyRepo.mock, checker); + return new StorageProviderConfig(props, repo, keyRepo.mock, checker, audit); } } diff --git a/app/proprietary/src/test/java/stirling/software/proprietary/storage/controller/FileStorageControllerMoreTest.java b/app/proprietary/src/test/java/stirling/software/proprietary/storage/controller/FileStorageControllerMoreTest.java index 1575dfbff5..f423ecba22 100644 --- a/app/proprietary/src/test/java/stirling/software/proprietary/storage/controller/FileStorageControllerMoreTest.java +++ b/app/proprietary/src/test/java/stirling/software/proprietary/storage/controller/FileStorageControllerMoreTest.java @@ -31,6 +31,7 @@ import org.springframework.web.multipart.MultipartFile; import org.springframework.web.server.ResponseStatusException; import stirling.software.proprietary.security.model.User; +import stirling.software.proprietary.service.AuditService; import stirling.software.proprietary.storage.model.FileShare; import stirling.software.proprietary.storage.model.ShareAccessRole; import stirling.software.proprietary.storage.model.StoredFile; @@ -48,12 +49,13 @@ class FileStorageControllerMoreTest { @Mock private FileStorageService fileStorageService; @Mock private StorageProvider storageProvider; + @Mock private AuditService auditService; private FileStorageController controller; @BeforeEach void setUp() { - controller = new FileStorageController(fileStorageService, storageProvider); + controller = new FileStorageController(fileStorageService, storageProvider, auditService); } private User user() { diff --git a/app/proprietary/src/test/java/stirling/software/proprietary/storage/controller/FileStorageControllerTest.java b/app/proprietary/src/test/java/stirling/software/proprietary/storage/controller/FileStorageControllerTest.java index 79033abc2d..df6cad881d 100644 --- a/app/proprietary/src/test/java/stirling/software/proprietary/storage/controller/FileStorageControllerTest.java +++ b/app/proprietary/src/test/java/stirling/software/proprietary/storage/controller/FileStorageControllerTest.java @@ -1,11 +1,13 @@ package stirling.software.proprietary.storage.controller; +import static java.nio.charset.StandardCharsets.UTF_8; import static org.assertj.core.api.Assertions.assertThat; import static org.mockito.ArgumentMatchers.any; import static org.mockito.ArgumentMatchers.anyBoolean; import static org.mockito.ArgumentMatchers.anyString; import static org.mockito.ArgumentMatchers.eq; import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.verifyNoInteractions; import static org.mockito.Mockito.when; import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get; import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.header; @@ -14,20 +16,26 @@ import static org.springframework.test.web.servlet.result.MockMvcResultMatchers. import java.net.URI; import java.time.Duration; +import java.util.List; +import java.util.Map; import java.util.Optional; import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Test; import org.junit.jupiter.api.extension.ExtendWith; +import org.mockito.ArgumentCaptor; import org.mockito.Mock; import org.mockito.junit.jupiter.MockitoExtension; +import org.springframework.core.io.ByteArrayResource; import org.springframework.http.HttpHeaders; import org.springframework.http.HttpStatus; import org.springframework.test.web.servlet.MockMvc; import org.springframework.test.web.servlet.MvcResult; import org.springframework.test.web.servlet.setup.MockMvcBuilders; +import stirling.software.proprietary.audit.AuditEventType; import stirling.software.proprietary.security.model.User; +import stirling.software.proprietary.service.AuditService; import stirling.software.proprietary.storage.model.StoredFile; import stirling.software.proprietary.storage.provider.StorageProvider; import stirling.software.proprietary.storage.service.FileStorageService; @@ -40,13 +48,14 @@ class FileStorageControllerTest { @Mock private FileStorageService fileStorageService; @Mock private StorageProvider storageProvider; + @Mock private AuditService auditService; private MockMvc mockMvc; @BeforeEach void setUp() { FileStorageController controller = - new FileStorageController(fileStorageService, storageProvider); + new FileStorageController(fileStorageService, storageProvider, auditService); mockMvc = MockMvcBuilders.standaloneSetup(controller).build(); } @@ -113,6 +122,75 @@ class FileStorageControllerTest { eq("11/abc-doc.pdf"), any(Duration.class), eq(true), eq("doc.pdf")); } + @Test + void downloadFile_encryptedContent_auditsPlaintextExportAsAttachment() throws Exception { + StoredFile file = newStoredFile(); + file.setEncryptionKeyId("cafe1234-0000-0000-0000-000000000001"); + streamedDownload(file); + + mockMvc.perform(get("/api/v1/storage/files/{fileId}/download", 77L)) + .andExpect(status().isOk()); + + // The marker is the compliance evidence that a decrypted copy left the platform, so it + // must carry the key it came from and whether it was viewed in-app or saved. + assertThat(exportEvents()) + .singleElement() + .satisfies( + event -> { + assertThat(event).containsEntry("action", "plaintextExport"); + assertThat(event).containsEntry("fileId", 77L); + assertThat(event).containsEntry("inline", false); + assertThat(event) + .containsEntry("keyId", "cafe1234-0000-0000-0000-000000000001"); + }); + } + + @Test + void downloadFile_inlineView_marksTheExportInline() throws Exception { + StoredFile file = newStoredFile(); + file.setEncryptionKeyId("cafe1234-0000-0000-0000-000000000001"); + streamedDownload(file); + + mockMvc.perform(get("/api/v1/storage/files/{fileId}/download", 77L).param("inline", "true")) + .andExpect(status().isOk()); + + // An in-app view and a saved copy are both exports, but a reviewer needs to tell them + // apart. + assertThat(exportEvents()) + .singleElement() + .satisfies(e -> assertThat(e).containsEntry("inline", true)); + } + + @Test + void downloadFile_plaintextContent_recordsNoExportEvent() throws Exception { + StoredFile file = newStoredFile(); // encryptionKeyId stays null + streamedDownload(file); + + mockMvc.perform(get("/api/v1/storage/files/{fileId}/download", 77L)) + .andExpect(status().isOk()); + + // Nothing was encrypted at rest, so there is no decryption to attest to. + verifyNoInteractions(auditService); + } + + /** Stubs an app-streamed (non-presigned) download of {@code file}. */ + private void streamedDownload(StoredFile file) throws Exception { + when(fileStorageService.requireAuthenticatedUser()).thenReturn(file.getOwner()); + when(fileStorageService.getAccessibleFile(file.getOwner(), 77L)).thenReturn(file); + when(storageProvider.signedDownloadUrl( + anyString(), any(Duration.class), anyBoolean(), anyString())) + .thenReturn(Optional.empty()); + when(fileStorageService.loadFile(file)) + .thenReturn(new ByteArrayResource("decrypted bytes".getBytes(UTF_8))); + } + + private List> exportEvents() { + @SuppressWarnings("unchecked") + ArgumentCaptor> data = ArgumentCaptor.forClass(Map.class); + verify(auditService).audit(eq(AuditEventType.STORAGE_ENCRYPTION), data.capture()); + return data.getAllValues(); + } + private static StoredFile newStoredFile() { User user = new User(); user.setId(11L); diff --git a/app/proprietary/src/test/java/stirling/software/proprietary/storage/controller/StorageEncryptionAdminControllerHttpTest.java b/app/proprietary/src/test/java/stirling/software/proprietary/storage/controller/StorageEncryptionAdminControllerHttpTest.java new file mode 100644 index 0000000000..27b5a893c7 --- /dev/null +++ b/app/proprietary/src/test/java/stirling/software/proprietary/storage/controller/StorageEncryptionAdminControllerHttpTest.java @@ -0,0 +1,209 @@ +package stirling.software.proprietary.storage.controller; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; +import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get; +import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.post; +import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.jsonPath; +import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status; + +import java.time.Instant; +import java.util.Base64; +import java.util.Optional; +import java.util.UUID; + +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.springframework.security.access.prepost.PreAuthorize; +import org.springframework.test.web.servlet.MockMvc; +import org.springframework.test.web.servlet.setup.MockMvcBuilders; + +import stirling.software.common.model.ApplicationProperties; +import stirling.software.proprietary.model.Team; +import stirling.software.proprietary.security.model.User; +import stirling.software.proprietary.service.AuditService; +import stirling.software.proprietary.storage.crypto.FileEncryptionKeyService; +import stirling.software.proprietary.storage.crypto.FileEncryptionMasterKey; +import stirling.software.proprietary.storage.crypto.InMemoryKeyRepo; +import stirling.software.proprietary.storage.crypto.StorageEncryptionAuditListener; +import stirling.software.proprietary.storage.crypto.StorageEncryptionState; +import stirling.software.proprietary.storage.repository.StoredFileRepository; +import stirling.software.proprietary.storage.service.StorageEncryptionMigrationService; + +/** + * Drives the admin API over HTTP rather than as Java calls: path mappings, status codes and — the + * part unit tests can't see — that the response records actually serialise. A UUID/Instant/enum + * that Jackson cannot render would pass every direct-invocation test and fail the first request. + * + *

Note {@code standaloneSetup} deliberately does not install the security filter chain, so this + * cannot prove {@code @PreAuthorize} enforcement; {@link #controller_isAdminOnly()} pins the + * annotation instead (method security is enabled globally in {@code SecurityConfiguration}). + */ +class StorageEncryptionAdminControllerHttpTest { + + private static final String MASTER = + Base64.getEncoder().encodeToString("0123456789abcdef0123456789abcdef".getBytes()); + + private InMemoryKeyRepo keyRepo; + private FileEncryptionKeyService keyService; + private StorageEncryptionMigrationService migrationService; + private MockMvc mockMvc; + + @BeforeEach + void setUp() { + keyRepo = new InMemoryKeyRepo(); + keyService = + new FileEncryptionKeyService( + keyRepo.mock, new FileEncryptionMasterKey(MASTER, false)); + StoredFileRepository storedFileRepository = mock(StoredFileRepository.class); + when(storedFileRepository.countByEncryptionKeyIdIsNotNull()).thenReturn(7L); + when(storedFileRepository.countByEncryptionKeyIdIsNull()).thenReturn(3L); + migrationService = mock(StorageEncryptionMigrationService.class); + + ApplicationProperties props = new ApplicationProperties(); + props.getStorage().setEnabled(true); + StorageEncryptionAdminController controller = + new StorageEncryptionAdminController( + props, + StorageEncryptionState.of( + true, keyService, StorageEncryptionAuditListener.NOOP), + keyRepo.mock, + storedFileRepository, + migrationService, + mock(AuditService.class)); + mockMvc = MockMvcBuilders.standaloneSetup(controller).build(); + } + + private UUID seedKey() throws Exception { + Team team = new Team(); + team.setId(1L); + User owner = new User(); + owner.setTeam(team); + return keyService.activeKekForOwner(owner).keyId(); + } + + @Test + void status_serialisesCountsFingerprintAndKeyRows() throws Exception { + UUID keyId = seedKey(); + + mockMvc.perform(get("/api/v1/admin/storage-encryption/status")) + .andExpect(status().isOk()) + .andExpect(jsonPath("$.writeEnabled").value(true)) + .andExpect(jsonPath("$.active").value(true)) + .andExpect(jsonPath("$.masterKeyFingerprint").isString()) + .andExpect(jsonPath("$.masterKeyVersion").value(1)) + .andExpect(jsonPath("$.encryptedFiles").value(7)) + .andExpect(jsonPath("$.plaintextFiles").value(3)) + .andExpect(jsonPath("$.keys[0].keyId").value(keyId.toString())) + .andExpect(jsonPath("$.keys[0].scopeType").value("TEAM")) + .andExpect(jsonPath("$.keys[0].scopeId").value(1)) + .andExpect(jsonPath("$.keys[0].status").value("ACTIVE")); + } + + @Test + void disableThenEnable_serialiseKeyInfoAndReportStatusTransitions() throws Exception { + UUID keyId = seedKey(); + + mockMvc.perform(post("/api/v1/admin/storage-encryption/keys/" + keyId + "/disable")) + .andExpect(status().isOk()) + .andExpect(jsonPath("$.keyId").value(keyId.toString())) + .andExpect(jsonPath("$.status").value("DISABLED")) + .andExpect(jsonPath("$.statusChangedAt").isNotEmpty()); + + mockMvc.perform(post("/api/v1/admin/storage-encryption/keys/" + keyId + "/enable")) + .andExpect(status().isOk()) + .andExpect(jsonPath("$.status").value("ACTIVE")); + } + + @Test + void enable_onActiveKey_returns409() throws Exception { + UUID keyId = seedKey(); + + mockMvc.perform(post("/api/v1/admin/storage-encryption/keys/" + keyId + "/enable")) + .andExpect(status().isConflict()); + } + + @Test + void disable_unknownKey_returns404() throws Exception { + mockMvc.perform( + post( + "/api/v1/admin/storage-encryption/keys/" + + UUID.randomUUID() + + "/disable")) + .andExpect(status().isNotFound()); + } + + @Test + void migrateStatus_neverStarted_serialisesIdle() throws Exception { + when(migrationService.status()).thenReturn(Optional.empty()); + + mockMvc.perform(get("/api/v1/admin/storage-encryption/migrate/status")) + .andExpect(status().isOk()) + .andExpect(jsonPath("$.state").value("IDLE")); + } + + @Test + void migrate_serialisesRunningSnapshotIncludingInstants() throws Exception { + when(migrationService.start()) + .thenReturn( + new StorageEncryptionMigrationService.MigrationStatus( + StorageEncryptionMigrationService.State.RUNNING, + 10, + 2, + 1, + 0, + Instant.parse("2026-01-01T00:00:00Z"), + null)); + + mockMvc.perform(post("/api/v1/admin/storage-encryption/migrate")) + .andExpect(status().isOk()) + .andExpect(jsonPath("$.state").value("RUNNING")) + .andExpect(jsonPath("$.total").value(10)) + .andExpect(jsonPath("$.processed").value(2)) + .andExpect(jsonPath("$.skipped").value(1)) + .andExpect(jsonPath("$.startedAt").isNotEmpty()) + .andExpect(jsonPath("$.finishedAt").doesNotExist()); + } + + @Test + void migrate_alreadyRunning_returns409() throws Exception { + when(migrationService.start()).thenThrow(new IllegalStateException("already running")); + + mockMvc.perform(post("/api/v1/admin/storage-encryption/migrate")) + .andExpect(status().isConflict()); + } + + @Test + void rotate_serialisesRewrapCount() throws Exception { + UUID keyId = seedKey(); + keyRepo.rows.get(keyId).setMasterKeyVersion(0); + + mockMvc.perform(post("/api/v1/admin/storage-encryption/master/rotate")) + .andExpect(status().isOk()) + .andExpect(jsonPath("$.rewrapped").value(1)) + .andExpect(jsonPath("$.masterKeyVersion").value(1)); + } + + @Test + void controller_isAdminOnly() { + PreAuthorize preAuthorize = + StorageEncryptionAdminController.class.getAnnotation(PreAuthorize.class); + assertThat(preAuthorize).as("admin API must be role-gated").isNotNull(); + assertThat(preAuthorize.value()).isEqualTo("hasRole('ADMIN')"); + } + + @Test + void noDeleteEndpointExists() { + // The "key material can be disabled but never destroyed" guarantee is structural: assert no + // handler method maps a DELETE. + boolean anyDelete = + java.util.Arrays.stream(StorageEncryptionAdminController.class.getMethods()) + .anyMatch( + m -> + m.isAnnotationPresent( + org.springframework.web.bind.annotation + .DeleteMapping.class)); + assertThat(anyDelete).as("no endpoint may delete key material").isFalse(); + } +} diff --git a/app/proprietary/src/test/java/stirling/software/proprietary/storage/controller/StorageEncryptionAdminControllerTest.java b/app/proprietary/src/test/java/stirling/software/proprietary/storage/controller/StorageEncryptionAdminControllerTest.java new file mode 100644 index 0000000000..b976985d45 --- /dev/null +++ b/app/proprietary/src/test/java/stirling/software/proprietary/storage/controller/StorageEncryptionAdminControllerTest.java @@ -0,0 +1,228 @@ +package stirling.software.proprietary.storage.controller; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.assertThatThrownBy; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.never; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.verifyNoInteractions; +import static org.mockito.Mockito.when; + +import java.util.Base64; +import java.util.Optional; +import java.util.UUID; + +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.springframework.dao.InvalidDataAccessResourceUsageException; +import org.springframework.data.domain.Sort; +import org.springframework.http.HttpStatus; +import org.springframework.web.server.ResponseStatusException; + +import stirling.software.common.model.ApplicationProperties; +import stirling.software.proprietary.model.Team; +import stirling.software.proprietary.security.model.User; +import stirling.software.proprietary.service.AuditService; +import stirling.software.proprietary.storage.crypto.FileEncryptionKeyService; +import stirling.software.proprietary.storage.crypto.FileEncryptionMasterKey; +import stirling.software.proprietary.storage.crypto.InMemoryKeyRepo; +import stirling.software.proprietary.storage.crypto.StorageEncryptionAuditListener; +import stirling.software.proprietary.storage.crypto.StorageEncryptionState; +import stirling.software.proprietary.storage.model.api.StorageEncryptionStatusResponse; +import stirling.software.proprietary.storage.repository.FileEncryptionKeyRepository; +import stirling.software.proprietary.storage.repository.StoredFileRepository; +import stirling.software.proprietary.storage.service.StorageEncryptionMigrationService; + +class StorageEncryptionAdminControllerTest { + + private static final String MASTER = + Base64.getEncoder().encodeToString("0123456789abcdef0123456789abcdef".getBytes()); + + private InMemoryKeyRepo keyRepo; + private FileEncryptionKeyService keyService; + private StoredFileRepository storedFileRepository; + private StorageEncryptionMigrationService migrationService; + private StorageEncryptionAdminController controller; + + @BeforeEach + void setUp() { + keyRepo = new InMemoryKeyRepo(); + keyService = + new FileEncryptionKeyService( + keyRepo.mock, new FileEncryptionMasterKey(MASTER, false)); + storedFileRepository = mock(StoredFileRepository.class); + when(storedFileRepository.countByEncryptionKeyIdIsNotNull()).thenReturn(4L); + when(storedFileRepository.countByEncryptionKeyIdIsNull()).thenReturn(2L); + migrationService = mock(StorageEncryptionMigrationService.class); + controller = + new StorageEncryptionAdminController( + storageEnabled(true), + StorageEncryptionState.of( + true, keyService, StorageEncryptionAuditListener.NOOP), + keyRepo.mock, + storedFileRepository, + migrationService, + mock(AuditService.class)); + } + + private static ApplicationProperties storageEnabled(boolean enabled) { + ApplicationProperties props = new ApplicationProperties(); + props.getStorage().setEnabled(enabled); + return props; + } + + private UUID createKey() throws Exception { + Team team = new Team(); + team.setId(1L); + User owner = new User(); + owner.setTeam(team); + return keyService.activeKekForOwner(owner).keyId(); + } + + @Test + void status_reportsCountsFingerprintAndKeys() throws Exception { + UUID keyId = createKey(); + StorageEncryptionStatusResponse status = controller.status(); + + assertThat(status.writeEnabled()).isTrue(); + assertThat(status.active()).isTrue(); + assertThat(status.masterKeyFingerprint()).hasSize(16); + assertThat(status.encryptedFiles()).isEqualTo(4); + assertThat(status.plaintextFiles()).isEqualTo(2); + assertThat(status.keys()).hasSize(1); + assertThat(status.keys().get(0).keyId()).isEqualTo(keyId); + assertThat(status.keys().get(0).status()).isEqualTo("ACTIVE"); + } + + @Test + void disableThenEnable_roundTripsAndFailsClosedInBetween() throws Exception { + UUID keyId = createKey(); + + StorageEncryptionStatusResponse.KeyInfo disabled = controller.disableKey(keyId); + assertThat(disabled.status()).isEqualTo("DISABLED"); + assertThatThrownBy(() -> keyService.kekForDecrypt(keyId)).hasMessageContaining("disabled"); + + StorageEncryptionStatusResponse.KeyInfo enabled = controller.enableKey(keyId); + assertThat(enabled.status()).isEqualTo("ACTIVE"); + assertThat(keyService.kekForDecrypt(keyId)).isNotNull(); + } + + @Test + void enable_keyNotDisabled_conflicts() throws Exception { + UUID keyId = createKey(); + assertThatThrownBy(() -> controller.enableKey(keyId)) + .isInstanceOf(ResponseStatusException.class) + .satisfies( + e -> + assertThat(((ResponseStatusException) e).getStatusCode()) + .isEqualTo(HttpStatus.CONFLICT)); + } + + @Test + void disable_unknownKey_notFound() { + assertThatThrownBy(() -> controller.disableKey(UUID.randomUUID())) + .isInstanceOf(ResponseStatusException.class) + .satisfies( + e -> + assertThat(((ResponseStatusException) e).getStatusCode()) + .isEqualTo(HttpStatus.NOT_FOUND)); + } + + @Test + void migrate_alreadyRunning_conflicts() { + when(migrationService.start()).thenThrow(new IllegalStateException("already running")); + assertThatThrownBy(() -> controller.startMigration()) + .isInstanceOf(ResponseStatusException.class) + .satisfies( + e -> + assertThat(((ResponseStatusException) e).getStatusCode()) + .isEqualTo(HttpStatus.CONFLICT)); + } + + @Test + void migrationStatus_neverStarted_reportsIdle() { + when(migrationService.status()).thenReturn(Optional.empty()); + assertThat(controller.migrationStatus().state()).isEqualTo("IDLE"); + } + + @Test + void rotate_whenInactive_conflicts() { + StorageEncryptionAdminController inactive = + new StorageEncryptionAdminController( + storageEnabled(true), + new StorageEncryptionState( + false, + () -> { + throw new IllegalStateException("no key material configured"); + }, + keyRepo.mock, + StorageEncryptionAuditListener.NOOP), + keyRepo.mock, + storedFileRepository, + migrationService, + mock(AuditService.class)); + assertThatThrownBy(inactive::rotateMasterKey) + .isInstanceOf(ResponseStatusException.class) + .satisfies( + e -> + assertThat(((ResponseStatusException) e).getStatusCode()) + .isEqualTo(HttpStatus.CONFLICT)); + } + + @Test + void status_storageDisabled_refusesWithoutTouchingTheRegistry() { + StorageEncryptionAdminController storageOff = + new StorageEncryptionAdminController( + storageEnabled(false), + StorageEncryptionState.of( + false, keyService, StorageEncryptionAuditListener.NOOP), + keyRepo.mock, + storedFileRepository, + migrationService, + mock(AuditService.class)); + + assertThatThrownBy(storageOff::status) + .isInstanceOf(ResponseStatusException.class) + .satisfies( + e -> + assertThat(((ResponseStatusException) e).getStatusCode()) + .isEqualTo(HttpStatus.FORBIDDEN)); + // The registry table need not exist on a deployment that never stores files. + verifyNoInteractions(storedFileRepository); + verify(keyRepo.mock, never()).findAll(any(Sort.class)); + } + + @Test + void status_unreadableRegistry_reports503RatherThanRaw500() { + FileEncryptionKeyRepository broken = mock(FileEncryptionKeyRepository.class); + when(broken.findAll(any(Sort.class))) + .thenThrow(new InvalidDataAccessResourceUsageException("no such table")); + StorageEncryptionAdminController brokenRegistry = + new StorageEncryptionAdminController( + storageEnabled(true), + StorageEncryptionState.of( + true, keyService, StorageEncryptionAuditListener.NOOP), + broken, + storedFileRepository, + migrationService, + mock(AuditService.class)); + + assertThatThrownBy(brokenRegistry::status) + .isInstanceOf(ResponseStatusException.class) + .satisfies( + e -> + assertThat(((ResponseStatusException) e).getStatusCode()) + .isEqualTo(HttpStatus.SERVICE_UNAVAILABLE)); + } + + @Test + void rotate_rewrapsPendingRows() throws Exception { + UUID keyId = createKey(); + keyRepo.rows.get(keyId).setMasterKeyVersion(0); // pretend wrapped by an older master + + var result = controller.rotateMasterKey(); + assertThat(result.get("rewrapped")).isEqualTo(1); + assertThat(keyRepo.rows.get(keyId).getMasterKeyVersion()).isEqualTo(1); + } +} diff --git a/app/proprietary/src/test/java/stirling/software/proprietary/storage/crypto/AuditingStorageEncryptionListenerTest.java b/app/proprietary/src/test/java/stirling/software/proprietary/storage/crypto/AuditingStorageEncryptionListenerTest.java new file mode 100644 index 0000000000..0c5fa53bc7 --- /dev/null +++ b/app/proprietary/src/test/java/stirling/software/proprietary/storage/crypto/AuditingStorageEncryptionListenerTest.java @@ -0,0 +1,99 @@ +package stirling.software.proprietary.storage.crypto; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.mockito.ArgumentMatchers.anyMap; +import static org.mockito.ArgumentMatchers.eq; +import static org.mockito.Mockito.atLeastOnce; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.never; +import static org.mockito.Mockito.verify; + +import java.util.List; +import java.util.Map; +import java.util.UUID; + +import org.junit.jupiter.api.Test; +import org.mockito.ArgumentCaptor; + +import stirling.software.proprietary.audit.AuditEventType; +import stirling.software.proprietary.service.AuditService; + +/** + * Covers the one knob a compliance reviewer actually turns: {@code storage.encryption.auditReads} + * must silence per-read decrypt events without silencing denials or key lifecycle events. + */ +class AuditingStorageEncryptionListenerTest { + + private static final UUID KEY_ID = UUID.fromString("11111111-2222-3333-4444-555555555555"); + + private final AuditService auditService = mock(AuditService.class); + + private List> auditedEvents() { + @SuppressWarnings("unchecked") + ArgumentCaptor> data = ArgumentCaptor.forClass(Map.class); + verify(auditService, atLeastOnce()) + .audit(eq(AuditEventType.STORAGE_ENCRYPTION), data.capture()); + return data.getAllValues(); + } + + @Test + void auditReadsOn_recordsDecryptWithKeyAndStorageKey() { + new AuditingStorageEncryptionListener(auditService, true).decrypted("team/blob", KEY_ID); + + assertThat(auditedEvents()) + .singleElement() + .satisfies( + event -> { + assertThat(event).containsEntry("action", "decrypt"); + assertThat(event).containsEntry("storageKey", "team/blob"); + assertThat(event).containsEntry("keyId", KEY_ID.toString()); + }); + } + + @Test + void auditReadsOff_dropsDecryptButKeepsEverythingElse() { + AuditingStorageEncryptionListener listener = + new AuditingStorageEncryptionListener(auditService, false); + + listener.decrypted("team/blob", KEY_ID); + verify(auditService, never()).audit(eq(AuditEventType.STORAGE_ENCRYPTION), anyMap()); + + listener.encrypted("team/blob", KEY_ID); + listener.decryptDenied(KEY_ID, "key disabled"); + listener.keyCreated(KEY_ID, "TEAM:7", 2); + + // Denials and lifecycle events are the compliance-relevant ones; they are never optional. + assertThat(auditedEvents()) + .extracting(event -> event.get("action")) + .containsExactly("encrypt", "decrypt.denied", "key.created"); + } + + @Test + void decryptDenied_carriesTheReason() { + new AuditingStorageEncryptionListener(auditService, false) + .decryptDenied(KEY_ID, "key not found"); + + assertThat(auditedEvents()) + .singleElement() + .satisfies( + event -> { + assertThat(event).containsEntry("action", "decrypt.denied"); + assertThat(event).containsEntry("reason", "key not found"); + assertThat(event).containsEntry("keyId", KEY_ID.toString()); + }); + } + + @Test + void keyCreated_carriesScopeAndVersion() { + new AuditingStorageEncryptionListener(auditService, true).keyCreated(KEY_ID, "TEAM:7", 2); + + assertThat(auditedEvents()) + .singleElement() + .satisfies( + event -> { + assertThat(event).containsEntry("action", "key.created"); + assertThat(event).containsEntry("scope", "TEAM:7"); + assertThat(event).containsEntry("keyVersion", 2); + }); + } +} diff --git a/app/proprietary/src/test/java/stirling/software/proprietary/storage/crypto/EncryptingStorageProviderTest.java b/app/proprietary/src/test/java/stirling/software/proprietary/storage/crypto/EncryptingStorageProviderTest.java index f66d03ae42..ce592f8bb0 100644 --- a/app/proprietary/src/test/java/stirling/software/proprietary/storage/crypto/EncryptingStorageProviderTest.java +++ b/app/proprietary/src/test/java/stirling/software/proprietary/storage/crypto/EncryptingStorageProviderTest.java @@ -10,6 +10,7 @@ import java.nio.file.Files; import java.nio.file.Path; import java.time.Duration; import java.util.Base64; +import java.util.List; import java.util.UUID; import org.junit.jupiter.api.BeforeEach; @@ -239,6 +240,54 @@ class EncryptingStorageProviderTest { } } + @Test + void auditListener_receivesEncryptDecryptAndDeniedEvents() throws IOException { + List events = new java.util.ArrayList<>(); + StorageEncryptionAuditListener recording = + new StorageEncryptionAuditListener() { + @Override + public void encrypted(String storageKey, UUID keyId) { + events.add("encrypt:" + storageKey); + } + + @Override + public void decrypted(String storageKey, UUID keyId) { + events.add("decrypt:" + storageKey); + } + + @Override + public void decryptDenied(UUID keyId, String reason) { + events.add("denied:" + reason); + } + }; + FileEncryptionKeyService keyService = + new FileEncryptionKeyService( + repo.mock, new FileEncryptionMasterKey(MASTER, false), recording); + EncryptingStorageProvider audited = + new EncryptingStorageProvider(inner, keyService, true, recording); + + StoredObject stored = audited.store(owner, upload()); + try (InputStream in = audited.load(stored.getStorageKey()).getInputStream()) { + in.readAllBytes(); + } + // Legacy plaintext must NOT produce a decrypt event. + StoredObject legacy = inner.store(owner, upload()); + audited.load(legacy.getStorageKey()); + + assertThat(events) + .containsExactly( + "encrypt:" + stored.getStorageKey(), "decrypt:" + stored.getStorageKey()); + + repo.rows + .get(UUID.fromString(stored.getEncryptionKeyId())) + .setStatus(FileEncryptionKey.Status.DISABLED); + keyService.invalidate(UUID.fromString(stored.getEncryptionKeyId())); + assertThatThrownBy(() -> audited.load(stored.getStorageKey())) + .isInstanceOf(StorageKeyRevokedException.class); + assertThat(events).hasSize(3); + assertThat(events.get(2)).isEqualTo("denied:key disabled"); + } + @Test void load_tinyLegacyBlob_shorterThanHeader_passesThrough() throws IOException { byte[] tiny = "hi".getBytes(StandardCharsets.UTF_8); @@ -347,7 +396,11 @@ class EncryptingStorageProviderTest { void signedDownloadUrl_delegatesWhenNoEncryptedContentPossible() throws IOException { // Vanilla install: flag off, no key rows anywhere -> keep the backend's fast path. StorageEncryptionState vanilla = - new StorageEncryptionState(false, () -> newKeyService(), repo.mock); + new StorageEncryptionState( + false, + () -> newKeyService(), + repo.mock, + StorageEncryptionAuditListener.NOOP); StorageProvider withUrls = new StorageProvider() { @Override @@ -383,7 +436,11 @@ class EncryptingStorageProviderTest { StoredObject encrypted = provider.store(owner, upload()); assertThat(encrypted.getEncryptionKeyId()).isNotNull(); StorageEncryptionState drifted = - new StorageEncryptionState(false, () -> newKeyService(), repo.mock); + new StorageEncryptionState( + false, + () -> newKeyService(), + repo.mock, + StorageEncryptionAuditListener.NOOP); EncryptingStorageProvider driftedNode = new EncryptingStorageProvider(withUrls, drifted); assertThat(driftedNode.signedDownloadUrl("k", Duration.ofMinutes(5), false, "a.pdf")) .isEmpty(); diff --git a/app/proprietary/src/test/java/stirling/software/proprietary/storage/crypto/FileEncryptionKeyServiceTest.java b/app/proprietary/src/test/java/stirling/software/proprietary/storage/crypto/FileEncryptionKeyServiceTest.java index 10c0a1431d..a4dd9720d3 100644 --- a/app/proprietary/src/test/java/stirling/software/proprietary/storage/crypto/FileEncryptionKeyServiceTest.java +++ b/app/proprietary/src/test/java/stirling/software/proprietary/storage/crypto/FileEncryptionKeyServiceTest.java @@ -6,7 +6,9 @@ import static org.mockito.ArgumentMatchers.any; import static org.mockito.Mockito.doAnswer; import static org.mockito.Mockito.doThrow; +import java.nio.charset.StandardCharsets; import java.util.Base64; +import java.util.List; import java.util.UUID; import org.junit.jupiter.api.BeforeEach; @@ -143,6 +145,56 @@ class FileEncryptionKeyServiceTest { .hasMessageContaining("cannot unwrap"); } + @Test + void verifyMasterKey_halfRotatedRegistryWithPreviousKeyRemoved_refusesStartup() + throws Exception { + // Team 1 gets re-wrapped under the new master key; team 2 does not, because the rotation + // stopped part-way (or was never run). + FileEncryptionKeyService.ScopeKek rewrapped = service.activeKekForOwner(teamUser(1)); + FileEncryptionKeyService.ScopeKek leftBehind = service.activeKekForOwner(teamUser(2)); + FileEncryptionMasterKey rotating = + new FileEncryptionMasterKey(MASTER_B, MASTER_A, 2, false); + FileEncryptionKey row = repo.rows.get(rewrapped.keyId()); + row.setWrappedKey( + Base64.getEncoder() + .encodeToString( + rotating.wrap( + rewrapped.key(), + rewrapped + .keyId() + .toString() + .getBytes(StandardCharsets.US_ASCII)))); + row.setMasterKeyVersion(2); + + // The operator now removes the previous key, believing rotation finished. Team 2's files + // are only recoverable while MASTER_A still exists, so startup must not look healthy. + FileEncryptionKeyService afterRemoval = + new FileEncryptionKeyService( + repo.mock, new FileEncryptionMasterKey(MASTER_B, null, 2, false)); + + assertThatThrownBy(afterRemoval::verifyMasterKey) + .isInstanceOf(IllegalStateException.class) + .hasMessageContaining("cannot unwrap 1 of 2") + .hasMessageContaining(leftBehind.keyId().toString()) + .hasMessageContaining("Refusing to start"); + } + + @Test + void verifyMasterKey_disabledRowThatCannotUnwrap_refusesStartup() throws Exception { + // Revocation is advertised as reversible, so an unreadable DISABLED row is just as much a + // loss of access as an unreadable ACTIVE one - it would not come back on enable. + FileEncryptionKeyService.ScopeKek created = service.activeKekForOwner(teamUser(1)); + service.setKeyStatus(created.keyId(), FileEncryptionKey.Status.DISABLED, "admin"); + + FileEncryptionKeyService wrongKey = + new FileEncryptionKeyService( + repo.mock, new FileEncryptionMasterKey(MASTER_B, false)); + + assertThatThrownBy(wrongKey::verifyMasterKey) + .isInstanceOf(IllegalStateException.class) + .hasMessageContaining("cannot unwrap 1 of 1"); + } + @Test void createActive_concurrentInsertRace_fallsBackToWinnersRow() throws Exception { // First save call hits the unique constraint; the service must re-read the winner's row. @@ -186,6 +238,111 @@ class FileEncryptionKeyServiceTest { assertThat(resolved.key()).isEqualTo(winnerKek); } + @Test + void setKeyStatus_disable_takesEffectImmediatelyOnSameService() throws Exception { + FileEncryptionKeyService.ScopeKek created = service.activeKekForOwner(teamUser(1)); + // The unwrap cache is warm from creation; disabling must invalidate it, not wait for TTL. + service.setKeyStatus(created.keyId(), FileEncryptionKey.Status.DISABLED, "admin"); + assertThatThrownBy(() -> service.kekForDecrypt(created.keyId())) + .isInstanceOf(StorageKeyRevokedException.class); + assertThat(repo.rows.get(created.keyId()).getStatusChangedBy()).isEqualTo("admin"); + + service.setKeyStatus(created.keyId(), FileEncryptionKey.Status.ACTIVE, "admin"); + assertThat(service.kekForDecrypt(created.keyId())).isEqualTo(created.key()); + } + + @Test + void unwrap_fallsBackToPreviousMasterKeyDuringRotation() throws Exception { + FileEncryptionKeyService.ScopeKek created = service.activeKekForOwner(teamUser(1)); + + // New primary key B, old key A kept as previous, version bumped to 2. + FileEncryptionMasterKey rotated = new FileEncryptionMasterKey(MASTER_B, MASTER_A, 2, false); + FileEncryptionKeyService rotatedService = new FileEncryptionKeyService(repo.mock, rotated); + assertThat(rotatedService.kekForDecrypt(created.keyId())).isEqualTo(created.key()); + rotatedService.verifyMasterKey(); // must pass via the previous-key fallback + } + + @Test + void rotateMasterKey_rewrapsRowsBelowCurrentVersion() throws Exception { + FileEncryptionKeyService.ScopeKek created = service.activeKekForOwner(teamUser(1)); + String wrappedBefore = repo.rows.get(created.keyId()).getWrappedKey(); + + FileEncryptionMasterKey rotated = new FileEncryptionMasterKey(MASTER_B, MASTER_A, 2, false); + FileEncryptionKeyService rotatedService = new FileEncryptionKeyService(repo.mock, rotated); + assertThat(rotatedService.rotateMasterKey()).isEqualTo(1); + + FileEncryptionKey row = repo.rows.get(created.keyId()); + assertThat(row.getMasterKeyVersion()).isEqualTo(2); + assertThat(row.getWrappedKey()).isNotEqualTo(wrappedBefore); + + // Same key material now unwraps WITHOUT the previous key configured. + FileEncryptionKeyService afterCleanup = + new FileEncryptionKeyService( + repo.mock, new FileEncryptionMasterKey(MASTER_B, null, 2, false)); + assertThat(afterCleanup.kekForDecrypt(created.keyId())).isEqualTo(created.key()); + + // Second rotation call is a no-op. + assertThat(rotatedService.rotateMasterKey()).isZero(); + } + + @Test + void enable_afterScopeMintedAnotherKey_comesBackRetiredNotSecondActive() throws Exception { + FileEncryptionKeyService.ScopeKek revoked = service.activeKekForOwner(teamUser(1)); + service.setKeyStatus(revoked.keyId(), FileEncryptionKey.Status.DISABLED, "admin"); + + // Revoking does not stop the team uploading: this mints a second key for the scope. + FileEncryptionKeyService.ScopeKek minted = service.activeKekForOwner(teamUser(1)); + assertThat(minted.keyId()).isNotEqualTo(revoked.keyId()); + + FileEncryptionKey reEnabled = service.enable(revoked.keyId(), "admin"); + + assertThat(reEnabled.getStatus()).isEqualTo(FileEncryptionKey.Status.RETIRED); + assertThat(activeKeysForTeam(1)).containsExactly(minted.keyId()); + // RETIRED still unwraps, so the revoked content is readable again. + assertThat(service.kekForDecrypt(revoked.keyId())).isEqualTo(revoked.key()); + } + + @Test + void enable_scopeHasNoOtherActiveKey_comesBackActive() throws Exception { + FileEncryptionKeyService.ScopeKek created = service.activeKekForOwner(teamUser(1)); + service.setKeyStatus(created.keyId(), FileEncryptionKey.Status.DISABLED, "admin"); + + FileEncryptionKey reEnabled = service.enable(created.keyId(), "admin"); + + assertThat(reEnabled.getStatus()).isEqualTo(FileEncryptionKey.Status.ACTIVE); + assertThat(activeKeysForTeam(1)).containsExactly(created.keyId()); + assertThat(service.kekForDecrypt(created.keyId())).isEqualTo(created.key()); + } + + @Test + void activeKekForOwner_twoActiveRowsInScope_picksTheHighestVersionOnEveryNode() + throws Exception { + // A registry left in the pre-fix shape (an older build re-enabled a key straight to + // ACTIVE). + FileEncryptionKeyService.ScopeKek older = service.activeKekForOwner(teamUser(1)); + service.setKeyStatus(older.keyId(), FileEncryptionKey.Status.DISABLED, "admin"); + FileEncryptionKeyService.ScopeKek newer = service.activeKekForOwner(teamUser(1)); + service.setKeyStatus(older.keyId(), FileEncryptionKey.Status.ACTIVE, "old-build"); + assertThat(activeKeysForTeam(1)).hasSize(2); + + // Two independent nodes (fresh caches) must agree, rather than follow DB row order. + for (int node = 0; node < 2; node++) { + FileEncryptionKeyService fresh = + new FileEncryptionKeyService( + repo.mock, new FileEncryptionMasterKey(MASTER_A, false)); + assertThat(fresh.activeKekForOwner(teamUser(1)).keyId()).isEqualTo(newer.keyId()); + } + } + + private List activeKeysForTeam(long teamId) { + return repo.rows.values().stream() + .filter(r -> r.getScopeType() == FileEncryptionKey.ScopeType.TEAM) + .filter(r -> r.getScopeId() == teamId) + .filter(r -> r.getStatus() == FileEncryptionKey.Status.ACTIVE) + .map(FileEncryptionKey::getKeyId) + .toList(); + } + @Test void createActive_raceWithoutWinner_rethrows() { doThrow(new DataIntegrityViolationException("duplicate key")) diff --git a/app/proprietary/src/test/java/stirling/software/proprietary/storage/crypto/InMemoryKeyRepo.java b/app/proprietary/src/test/java/stirling/software/proprietary/storage/crypto/InMemoryKeyRepo.java index cc3828333a..3d5e7ede3d 100644 --- a/app/proprietary/src/test/java/stirling/software/proprietary/storage/crypto/InMemoryKeyRepo.java +++ b/app/proprietary/src/test/java/stirling/software/proprietary/storage/crypto/InMemoryKeyRepo.java @@ -1,16 +1,20 @@ package stirling.software.proprietary.storage.crypto; import static org.mockito.ArgumentMatchers.any; +import static org.mockito.ArgumentMatchers.anyInt; import static org.mockito.ArgumentMatchers.anyLong; import static org.mockito.Mockito.mock; import static org.mockito.Mockito.when; import java.util.Comparator; +import java.util.List; import java.util.Map; import java.util.Optional; import java.util.UUID; import java.util.concurrent.ConcurrentHashMap; +import org.springframework.data.domain.Sort; + import stirling.software.proprietary.storage.model.FileEncryptionKey; import stirling.software.proprietary.storage.repository.FileEncryptionKeyRepository; @@ -38,14 +42,17 @@ public final class InMemoryKeyRepo { }); when(mock.findById(any(UUID.class))) .thenAnswer(inv -> Optional.ofNullable(rows.get(inv.getArgument(0)))); - when(mock.findFirstByScopeTypeAndScopeIdAndStatus(any(), anyLong(), any())) + when(mock.findFirstByScopeTypeAndScopeIdAndStatusOrderByKeyVersionDesc( + any(), anyLong(), any())) .thenAnswer( inv -> rows.values().stream() .filter(r -> r.getScopeType() == inv.getArgument(0)) .filter(r -> r.getScopeId() == inv.getArgument(1)) .filter(r -> r.getStatus() == inv.getArgument(2)) - .findFirst()); + .max( + Comparator.comparingInt( + FileEncryptionKey::getKeyVersion))); when(mock.findFirstByScopeTypeAndScopeIdOrderByKeyVersionDesc(any(), anyLong())) .thenAnswer( inv -> @@ -62,5 +69,34 @@ public final class InMemoryKeyRepo { .filter(r -> r.getStatus() == inv.getArgument(0)) .findFirst()); when(mock.count()).thenAnswer(inv -> (long) rows.size()); + when(mock.findAll()).thenAnswer(inv -> List.copyOf(rows.values())); + when(mock.countByMasterKeyVersionLessThan(anyInt())) + .thenAnswer( + inv -> + rows.values().stream() + .filter( + r -> + r.getMasterKeyVersion() + < inv.getArgument(0)) + .count()); + when(mock.findByMasterKeyVersionLessThan(anyInt())) + .thenAnswer( + inv -> + rows.values().stream() + .filter( + r -> + r.getMasterKeyVersion() + < inv.getArgument(0)) + .toList()); + when(mock.countByMasterKeyVersionGreaterThan(anyInt())) + .thenAnswer( + inv -> + rows.values().stream() + .filter( + r -> + r.getMasterKeyVersion() + > inv.getArgument(0)) + .count()); + when(mock.findAll(any(Sort.class))).thenAnswer(inv -> List.copyOf(rows.values())); } } diff --git a/app/proprietary/src/test/java/stirling/software/proprietary/storage/repository/StoredFileMigrationQueriesDbTest.java b/app/proprietary/src/test/java/stirling/software/proprietary/storage/repository/StoredFileMigrationQueriesDbTest.java new file mode 100644 index 0000000000..430b0e1522 --- /dev/null +++ b/app/proprietary/src/test/java/stirling/software/proprietary/storage/repository/StoredFileMigrationQueriesDbTest.java @@ -0,0 +1,148 @@ +package stirling.software.proprietary.storage.repository; + +import static org.assertj.core.api.Assertions.assertThat; + +import java.util.List; +import java.util.UUID; + +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.SpringBootConfiguration; +import org.springframework.boot.autoconfigure.AutoConfigurationPackage; +import org.springframework.boot.data.jpa.test.autoconfigure.DataJpaTest; +import org.springframework.data.domain.PageRequest; +import org.springframework.test.annotation.DirtiesContext; + +import jakarta.persistence.EntityManager; + +import stirling.software.proprietary.model.Team; +import stirling.software.proprietary.security.model.User; +import stirling.software.proprietary.storage.model.StoredFile; + +/** + * The encrypt-existing migration's queries against a real database. The service-level tests mock + * {@link StoredFileRepository} wholesale, so without this the JOIN FETCH + Pageable selection and + * the three compare-and-swap updates would never actually execute — a wrong CAS predicate would + * pass every mocked test and then silently skip every file in production. + */ +@DataJpaTest +@DirtiesContext +class StoredFileMigrationQueriesDbTest { + + @Autowired private StoredFileRepository repository; + @Autowired private EntityManager entityManager; + + private User owner; + + @BeforeEach + void setUp() { + Team team = new Team(); + team.setName("team-" + UUID.randomUUID()); + entityManager.persist(team); + + owner = new User(); + owner.setUsername("owner-" + UUID.randomUUID()); + owner.setPassword("x"); + owner.setTeam(team); + entityManager.persist(owner); + entityManager.flush(); + } + + private StoredFile persistFile(String storageKey, String encryptionKeyId) { + StoredFile file = new StoredFile(); + file.setOwner(owner); + file.setOriginalFilename("doc.pdf"); + file.setContentType("application/pdf"); + file.setSizeBytes(123); + file.setStorageKey(storageKey); + file.setEncryptionKeyId(encryptionKeyId); + entityManager.persist(file); + entityManager.flush(); + return file; + } + + @Test + void findMigratableAfter_selectsOnlyPlaintextRows_inIdOrder_withOwnerFetched() { + StoredFile plain1 = persistFile("k-plain-1", null); + StoredFile plain2 = persistFile("k-plain-2", null); + persistFile("k-encrypted", UUID.randomUUID().toString()); + entityManager.clear(); + + List page = repository.findMigratableAfter(0L, PageRequest.of(0, 10)); + + assertThat(page) + .extracting(StoredFile::getId) + .containsExactly(plain1.getId(), plain2.getId()); + // JOIN FETCH must populate owner (+ its EAGER team) for use outside this session. + assertThat(page.get(0).getOwner().getUsername()).isEqualTo(owner.getUsername()); + assertThat(page.get(0).getOwner().getTeam()).isNotNull(); + } + + @Test + void findMigratableAfter_cursorAndPageSizeAreHonoured() { + StoredFile first = persistFile("k-1", null); + StoredFile second = persistFile("k-2", null); + entityManager.clear(); + + assertThat(repository.findMigratableAfter(0L, PageRequest.of(0, 1))) + .extracting(StoredFile::getId) + .containsExactly(first.getId()); + assertThat(repository.findMigratableAfter(first.getId(), PageRequest.of(0, 10))) + .extracting(StoredFile::getId) + .containsExactly(second.getId()); + assertThat(repository.findMigratableAfter(second.getId(), PageRequest.of(0, 10))).isEmpty(); + } + + @Test + void swapMainBlob_appliesOnlyWhenStorageKeyStillMatches() { + StoredFile file = persistFile("k-old", null); + String keyId = UUID.randomUUID().toString(); + entityManager.clear(); + + // Stale expectation (someone else replaced the file) -> no row updated. + assertThat(repository.swapMainBlob(file.getId(), "k-stale", "k-new", keyId)).isZero(); + + assertThat(repository.swapMainBlob(file.getId(), "k-old", "k-new", keyId)).isEqualTo(1); + entityManager.clear(); + + StoredFile reloaded = repository.findById(file.getId()).orElseThrow(); + assertThat(reloaded.getStorageKey()).isEqualTo("k-new"); + assertThat(reloaded.getEncryptionKeyId()).isEqualTo(keyId); + // Stamping the key id must remove the row from the migration's selection. + assertThat(repository.findMigratableAfter(0L, PageRequest.of(0, 10))).isEmpty(); + assertThat(repository.countByEncryptionKeyIdIsNotNull()).isEqualTo(1); + assertThat(repository.countByEncryptionKeyIdIsNull()).isZero(); + } + + @Test + void swapSecondaryBlobs_applyIndependentlyAndLeaveKeyIdAlone() { + StoredFile file = persistFile("k-main", null); + file.setHistoryStorageKey("k-hist-old"); + file.setAuditLogStorageKey("k-audit-old"); + entityManager.merge(file); + entityManager.flush(); + entityManager.clear(); + + assertThat(repository.swapHistoryBlob(file.getId(), "k-wrong", "k-hist-new")).isZero(); + assertThat(repository.swapHistoryBlob(file.getId(), "k-hist-old", "k-hist-new")) + .isEqualTo(1); + assertThat(repository.swapAuditLogBlob(file.getId(), "k-audit-old", "k-audit-new")) + .isEqualTo(1); + entityManager.clear(); + + StoredFile reloaded = repository.findById(file.getId()).orElseThrow(); + assertThat(reloaded.getHistoryStorageKey()).isEqualTo("k-hist-new"); + assertThat(reloaded.getAuditLogStorageKey()).isEqualTo("k-audit-new"); + // Secondary swaps must not stamp encryptionKeyId, so the row stays selectable until the + // main blob is done. + assertThat(reloaded.getEncryptionKeyId()).isNull(); + assertThat(repository.findMigratableAfter(0L, PageRequest.of(0, 10))) + .extracting(StoredFile::getId) + .containsExactly(file.getId()); + } + + @SpringBootConfiguration + @AutoConfigurationPackage(basePackages = "stirling.software.proprietary") + static class TestApp {} +} diff --git a/app/proprietary/src/test/java/stirling/software/proprietary/storage/service/StorageEncryptionMigrationServiceTest.java b/app/proprietary/src/test/java/stirling/software/proprietary/storage/service/StorageEncryptionMigrationServiceTest.java new file mode 100644 index 0000000000..e71f8ef714 --- /dev/null +++ b/app/proprietary/src/test/java/stirling/software/proprietary/storage/service/StorageEncryptionMigrationServiceTest.java @@ -0,0 +1,467 @@ +package stirling.software.proprietary.storage.service; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.assertThatThrownBy; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.ArgumentMatchers.anyLong; +import static org.mockito.ArgumentMatchers.anyString; +import static org.mockito.ArgumentMatchers.argThat; +import static org.mockito.ArgumentMatchers.eq; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.Base64; +import java.util.Comparator; +import java.util.Map; +import java.util.Optional; +import java.util.concurrent.ConcurrentHashMap; +import java.util.concurrent.atomic.AtomicBoolean; +import java.util.concurrent.atomic.AtomicInteger; + +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.io.TempDir; +import org.springframework.core.io.InputStreamResource; +import org.springframework.data.domain.Pageable; +import org.springframework.mock.web.MockMultipartFile; +import org.springframework.web.multipart.MultipartFile; + +import stirling.software.proprietary.audit.AuditEventType; +import stirling.software.proprietary.model.Team; +import stirling.software.proprietary.security.model.User; +import stirling.software.proprietary.service.AuditService; +import stirling.software.proprietary.storage.crypto.EncryptingStorageProvider; +import stirling.software.proprietary.storage.crypto.FileEncryptionKeyService; +import stirling.software.proprietary.storage.crypto.FileEncryptionMasterKey; +import stirling.software.proprietary.storage.crypto.InMemoryKeyRepo; +import stirling.software.proprietary.storage.crypto.StorageEncryptionAuditListener; +import stirling.software.proprietary.storage.crypto.StorageEncryptionState; +import stirling.software.proprietary.storage.model.StoredFile; +import stirling.software.proprietary.storage.provider.LocalStorageProvider; +import stirling.software.proprietary.storage.provider.StorageProvider; +import stirling.software.proprietary.storage.provider.StoredObject; +import stirling.software.proprietary.storage.repository.StoredFileRepository; + +class StorageEncryptionMigrationServiceTest { + + private static final byte[] CONTENT = + "legacy plaintext content that must end up encrypted".getBytes(StandardCharsets.UTF_8); + private static final String MASTER = + Base64.getEncoder() + .encodeToString( + "0123456789abcdef0123456789abcdef".getBytes(StandardCharsets.UTF_8)); + + @TempDir Path tempDir; + + private final Map rows = new ConcurrentHashMap<>(); + private LocalStorageProvider inner; + private StorageProvider provider; + private StoredFileRepository fileRepo; + private StorageEncryptionMigrationService service; + private FileEncryptionKeyService keyService; + private User owner; + + @BeforeEach + void setUp() { + inner = new LocalStorageProvider(tempDir); + keyService = + new FileEncryptionKeyService( + new InMemoryKeyRepo().mock, new FileEncryptionMasterKey(MASTER, false)); + provider = + new EncryptingStorageProvider( + inner, keyService, true, StorageEncryptionAuditListener.NOOP); + + Team team = new Team(); + team.setId(3L); + owner = new User(); + owner.setId(1L); + owner.setTeam(team); + + fileRepo = mock(StoredFileRepository.class); + when(fileRepo.countByEncryptionKeyIdIsNull()) + .thenAnswer( + inv -> + rows.values().stream() + .filter(f -> f.getEncryptionKeyId() == null) + .count()); + when(fileRepo.findMigratableAfter(anyLong(), any(Pageable.class))) + .thenAnswer( + inv -> { + long lastId = inv.getArgument(0); + Pageable p = inv.getArgument(1); + return rows.values().stream() + .filter(f -> f.getEncryptionKeyId() == null) + .filter(f -> f.getId() > lastId) + .sorted(Comparator.comparing(StoredFile::getId)) + .limit(p.getPageSize()) + .toList(); + }); + when(fileRepo.swapMainBlob(anyLong(), anyString(), anyString(), anyString())) + .thenAnswer( + inv -> { + StoredFile f = rows.get(inv.getArgument(0)); + if (f == null || !f.getStorageKey().equals(inv.getArgument(1))) { + return 0; + } + f.setStorageKey(inv.getArgument(2)); + f.setEncryptionKeyId(inv.getArgument(3)); + return 1; + }); + when(fileRepo.swapHistoryBlob(anyLong(), anyString(), anyString())) + .thenAnswer( + inv -> { + StoredFile f = rows.get(inv.getArgument(0)); + if (f == null || !inv.getArgument(1).equals(f.getHistoryStorageKey())) { + return 0; + } + f.setHistoryStorageKey(inv.getArgument(2)); + return 1; + }); + + service = + new StorageEncryptionMigrationService( + fileRepo, + provider, + StorageEncryptionState.of( + true, keyService, StorageEncryptionAuditListener.NOOP), + mock(AuditService.class)); + } + + private StoredFile plaintextFile(long id, boolean withHistory) throws Exception { + StoredObject main = + inner.store( + owner, + new MockMultipartFile( + "file", "doc" + id + ".pdf", "application/pdf", CONTENT)); + StoredFile f = new StoredFile(); + f.setId(id); + f.setOwner(owner); + f.setOriginalFilename("doc" + id + ".pdf"); + f.setContentType("application/pdf"); + f.setSizeBytes(CONTENT.length); + f.setStorageKey(main.getStorageKey()); + if (withHistory) { + StoredObject hist = + inner.store( + owner, + new MockMultipartFile( + "file", "hist" + id + ".zip", "application/zip", CONTENT)); + f.setHistoryStorageKey(hist.getStorageKey()); + f.setHistoryFilename("hist" + id + ".zip"); + f.setHistoryContentType("application/zip"); + f.setHistorySizeBytes((long) CONTENT.length); + } + rows.put(id, f); + return f; + } + + private StorageEncryptionMigrationService.MigrationStatus awaitCompletion() throws Exception { + for (int i = 0; i < 300; i++) { + Optional s = service.status(); + if (s.isPresent() + && s.get().state() != StorageEncryptionMigrationService.State.RUNNING) { + return s.get(); + } + Thread.sleep(100); + } + throw new AssertionError("migration did not finish in time"); + } + + @Test + void migrate_encryptsBacklogAndPreservesContent() throws Exception { + plaintextFile(1, false); + plaintextFile(2, true); + + service.start(); + StorageEncryptionMigrationService.MigrationStatus done = awaitCompletion(); + + assertThat(done.state()).isEqualTo(StorageEncryptionMigrationService.State.COMPLETED); + assertThat(done.processed()).isEqualTo(2); + assertThat(done.failed()).isZero(); + + for (StoredFile f : rows.values()) { + assertThat(f.getEncryptionKeyId()).isNotNull(); + byte[] onDisk = Files.readAllBytes(tempDir.resolve(f.getStorageKey())); + assertThat(new String(onDisk, 0, 8, StandardCharsets.US_ASCII)).isEqualTo("SPDFEAR1"); + try (var in = provider.load(f.getStorageKey()).getInputStream()) { + assertThat(in.readAllBytes()).isEqualTo(CONTENT); + } + if (f.getHistoryStorageKey() != null) { + byte[] hist = Files.readAllBytes(tempDir.resolve(f.getHistoryStorageKey())); + assertThat(new String(hist, 0, 8, StandardCharsets.US_ASCII)).isEqualTo("SPDFEAR1"); + } + } + // Exactly one blob per storage key remains (old plaintext blobs were deleted). + long blobCount; + try (var stream = Files.walk(tempDir)) { + blobCount = stream.filter(Files::isRegularFile).count(); + } + assertThat(blobCount).isEqualTo(3); + } + + @Test + void migrate_casMiss_discardsOwnCopyAndSkips() throws Exception { + plaintextFile(1, false); + // Simulate a user replacing the file mid-migration: main-blob CAS always misses. + when(fileRepo.swapMainBlob(anyLong(), anyString(), anyString(), anyString())).thenReturn(0); + + service.start(); + StorageEncryptionMigrationService.MigrationStatus done = awaitCompletion(); + + assertThat(done.skipped()).isEqualTo(1); + assertThat(done.processed()).isZero(); + StoredFile f = rows.get(1L); + assertThat(f.getEncryptionKeyId()).isNull(); + // The user's blob is untouched and still loadable; the migration's copy is gone. + byte[] onDisk = Files.readAllBytes(tempDir.resolve(f.getStorageKey())); + assertThat(onDisk).isEqualTo(CONTENT); + long blobCount; + try (var stream = Files.walk(tempDir)) { + blobCount = stream.filter(Files::isRegularFile).count(); + } + assertThat(blobCount).isEqualTo(1); + } + + @Test + void start_secondConcurrentRun_rejected() throws Exception { + for (long i = 1; i <= 30; i++) { + plaintextFile(i, false); + } + service.start(); + assertThatThrownBy(service::start) + .isInstanceOf(IllegalStateException.class) + .hasMessageContaining("already running"); + awaitCompletion(); + // After completion a new run is allowed again. + service.start(); + awaitCompletion(); + } + + @Test + void start_writeDisabled_rejected() { + StorageEncryptionMigrationService disabled = + new StorageEncryptionMigrationService( + fileRepo, + provider, + new StorageEncryptionState( + false, () -> null, null, StorageEncryptionAuditListener.NOOP), + mock(AuditService.class)); + assertThatThrownBy(disabled::start) + .isInstanceOf(IllegalStateException.class) + .hasMessageContaining("storage.encryption.enabled"); + } + + @Test + void migrate_writeFlagOffMidRun_stopsFailedWithoutChurningTheRestOfTheBacklog() + throws Exception { + for (long i = 1; i <= 30; i++) { // more than one page, so the run has somewhere to churn + plaintextFile(i, false); + } + AtomicBoolean writeEnabled = new AtomicBoolean(true); + AtomicInteger stores = new AtomicInteger(); + // Flip the flag the instant the first file has been stored, exactly as an admin toggling + // storage.encryption.enabled mid-run would. + StorageProvider countingInner = + new StorageProvider() { + @Override + public StoredObject store(User o, MultipartFile f) throws java.io.IOException { + StoredObject stored = inner.store(o, f); + if (stores.incrementAndGet() == 1) { + writeEnabled.set(false); + } + return stored; + } + + @Override + public org.springframework.core.io.Resource load(String key) + throws java.io.IOException { + return inner.load(key); + } + + @Override + public void delete(String key) throws java.io.IOException { + inner.delete(key); + } + }; + StorageEncryptionState flippable = flippableState(writeEnabled); + service = + new StorageEncryptionMigrationService( + fileRepo, + new EncryptingStorageProvider(countingInner, flippable), + flippable, + mock(AuditService.class)); + + service.start(); + StorageEncryptionMigrationService.MigrationStatus done = awaitCompletion(); + + assertThat(done.state()).isEqualTo(StorageEncryptionMigrationService.State.FAILED); + assertThat(done.processed()).isEqualTo(1); + // The remaining 29 files were never re-stored, so nothing was written and deleted for + // nothing, and none of them is counted as a failure. + assertThat(stores.get()).isEqualTo(1); + assertThat(done.failed()).isZero(); + assertThat(rows.values().stream().filter(f -> f.getEncryptionKeyId() == null).count()) + .isEqualTo(29); + } + + @Test + void start_auditsTheAdminWhoTriggeredIt_onStartAndCompletion() throws Exception { + plaintextFile(1, false); + AuditService audit = mock(AuditService.class); + when(audit.captureCurrentPrincipal()).thenReturn("admin-alice"); + service = + new StorageEncryptionMigrationService( + fileRepo, + provider, + StorageEncryptionState.of( + true, keyService, StorageEncryptionAuditListener.NOOP), + audit); + + service.start(); + awaitCompletion(); + + // Both ends of the run must name the admin: the completion event runs on a virtual thread + // with no security context, where the principal would otherwise resolve to "system". + verify(audit) + .audit( + eq("admin-alice"), + eq(AuditEventType.STORAGE_ENCRYPTION), + argThat(data -> "migration.started".equals(data.get("action")))); + verify(audit) + .audit( + eq("admin-alice"), + eq(AuditEventType.STORAGE_ENCRYPTION), + argThat(data -> "migration.completed".equals(data.get("action")))); + } + + // ---- one-shot backends (S3, database) ------------------------------------------------ + + /** + * Wraps the local provider so {@code load} hands back a stock {@link InputStreamResource}: a + * one-shot stream that refuses {@code contentLength()} once partially read, the way an S3 or + * database resource behaves. Local files are re-openable and never exercise that branch. + */ + private StorageProvider oneShotBacked() { + return new StorageProvider() { + @Override + public StoredObject store(User o, MultipartFile f) throws java.io.IOException { + return inner.store(o, f); + } + + @Override + public org.springframework.core.io.Resource load(String key) + throws java.io.IOException { + return new InputStreamResource(inner.load(key).getInputStream()); + } + + @Override + public void delete(String key) throws java.io.IOException { + inner.delete(key); + } + }; + } + + private void useProvider(StorageProvider backing) { + service = + new StorageEncryptionMigrationService( + fileRepo, + new EncryptingStorageProvider( + backing, + StorageEncryptionState.of( + true, keyService, StorageEncryptionAuditListener.NOOP)), + StorageEncryptionState.of( + true, keyService, StorageEncryptionAuditListener.NOOP), + mock(AuditService.class)); + } + + @Test + void migrate_oneShotResource_fallsBackToTheRecordedSizeAndStillEncrypts() throws Exception { + StoredFile file = plaintextFile(1, false); + useProvider(oneShotBacked()); + + service.start(); + StorageEncryptionMigrationService.MigrationStatus done = awaitCompletion(); + + assertThat(done.state()).isEqualTo(StorageEncryptionMigrationService.State.COMPLETED); + assertThat(done.processed()).isEqualTo(1); + assertThat(file.getEncryptionKeyId()).isNotNull(); + byte[] onDisk = Files.readAllBytes(tempDir.resolve(file.getStorageKey())); + assertThat(new String(onDisk, 0, 8, StandardCharsets.US_ASCII)).isEqualTo("SPDFEAR1"); + // Decrypting through a fresh load must return the original bytes, not a truncated stream. + try (var in = + new EncryptingStorageProvider(inner, keyService, true) + .load(file.getStorageKey()) + .getInputStream()) { + assertThat(in.readAllBytes()).isEqualTo(CONTENT); + } + } + + @Test + void migrate_oneShotResourceWithWrongRecordedSize_failsTheFileAndLeavesTheRowAlone() + throws Exception { + StoredFile file = plaintextFile(1, false); + String originalKey = file.getStorageKey(); + file.setSizeBytes(CONTENT.length + 7L); // stale/incorrect row size + useProvider(oneShotBacked()); + + service.start(); + StorageEncryptionMigrationService.MigrationStatus done = awaitCompletion(); + + // The size mismatch is caught inside store(), before the compare-and-swap, so the file is + // counted as failed and the row still points at its untouched plaintext blob. + assertThat(done.failed()).isEqualTo(1); + assertThat(done.processed()).isZero(); + assertThat(file.getEncryptionKeyId()).isNull(); + assertThat(file.getStorageKey()).isEqualTo(originalKey); + assertThat(Files.readAllBytes(tempDir.resolve(originalKey))).isEqualTo(CONTENT); + } + + @Test + void migrate_oneShotResourceWithNoRecordedSize_failsTheFileRatherThanGuessing() + throws Exception { + // Only the secondary blobs have a nullable size, so that is where "no size available at + // all" is reachable: neither the resource nor the row can say how long the plaintext is. + StoredFile file = plaintextFile(1, true); + String historyKey = file.getHistoryStorageKey(); + file.setHistorySizeBytes(null); + useProvider(oneShotBacked()); + + service.start(); + StorageEncryptionMigrationService.MigrationStatus done = awaitCompletion(); + + assertThat(done.failed()).isEqualTo(1); + assertThat(done.processed()).isZero(); + assertThat(file.getEncryptionKeyId()).isNull(); + assertThat(file.getHistoryStorageKey()).isEqualTo(historyKey); + } + + private StorageEncryptionState flippableState(AtomicBoolean writeEnabled) { + return new StorageEncryptionState( + true, () -> keyService, null, StorageEncryptionAuditListener.NOOP) { + @Override + public boolean isWriteEnabled() { + return writeEnabled.get(); + } + }; + } + + @Test + void migrate_perFileFailure_countsAndContinues() throws Exception { + StoredFile broken = plaintextFile(1, false); + plaintextFile(2, false); + // Point file 1 at a missing blob so its migration throws. + broken.setStorageKey("1/does-not-exist"); + + service.start(); + StorageEncryptionMigrationService.MigrationStatus done = awaitCompletion(); + + assertThat(done.state()).isEqualTo(StorageEncryptionMigrationService.State.COMPLETED); + assertThat(done.failed()).isEqualTo(1); + assertThat(done.processed()).isEqualTo(1); + assertThat(rows.get(2L).getEncryptionKeyId()).isNotNull(); + } +} diff --git a/app/proprietary/src/test/java/stirling/software/proprietary/workflow/service/WorkflowSessionServiceTest.java b/app/proprietary/src/test/java/stirling/software/proprietary/workflow/service/WorkflowSessionServiceTest.java index cf985d568e..184926574e 100644 --- a/app/proprietary/src/test/java/stirling/software/proprietary/workflow/service/WorkflowSessionServiceTest.java +++ b/app/proprietary/src/test/java/stirling/software/proprietary/workflow/service/WorkflowSessionServiceTest.java @@ -28,6 +28,7 @@ import stirling.software.common.model.ApplicationProperties.Storage; import stirling.software.common.model.ApplicationProperties.Storage.Signing; import stirling.software.proprietary.security.database.repository.UserRepository; import stirling.software.proprietary.security.model.User; +import stirling.software.proprietary.storage.crypto.StorageKeyRevokedException; import stirling.software.proprietary.storage.model.StoredFile; import stirling.software.proprietary.storage.provider.StorageProvider; import stirling.software.proprietary.storage.provider.StoredObject; @@ -639,4 +640,46 @@ class WorkflowSessionServiceTest { assertThat(service.listUserSessions(owner)).isEmpty(); } + + // ------------------------------------------------------------------------- + // revoked encryption keys (storage encryption kill switch) + // ------------------------------------------------------------------------- + + @Test + void getOriginalFile_revokedEncryptionKey_isForbiddenNotServerError() throws Exception { + WorkflowSession session = new WorkflowSession(); + StoredFile original = new StoredFile(); + original.setStorageKey("1/original"); + session.setOriginalFile(original); + when(workflowSessionRepository.findBySessionId("s1")).thenReturn(Optional.of(session)); + when(storageProvider.load("1/original")) + .thenThrow(new StorageKeyRevokedException("key disabled")); + + assertThatThrownBy(() -> service.getOriginalFile("s1")) + .isInstanceOf(ResponseStatusException.class) + .satisfies( + e -> + assertThat(((ResponseStatusException) e).getStatusCode()) + .isEqualTo(HttpStatus.FORBIDDEN)); + } + + @Test + void getSignRequestDocument_revokedEncryptionKey_isForbiddenNotServerError() throws Exception { + User participantUser = user("bob"); + WorkflowSession session = sessionWithParticipant("s2", pendingParticipant(participantUser)); + StoredFile original = new StoredFile(); + original.setStorageKey("1/sign-me"); + session.setOriginalFile(original); + when(storageProvider.load("1/sign-me")) + .thenThrow(new StorageKeyRevokedException("key disabled")); + + // This path wraps the read in catch(IOException) -> 500; the revocation must not be + // swallowed by it, because StorageKeyRevokedException IS an IOException. + assertThatThrownBy(() -> service.getSignRequestDocument("s2", participantUser)) + .isInstanceOf(ResponseStatusException.class) + .satisfies( + e -> + assertThat(((ResponseStatusException) e).getStatusCode()) + .isEqualTo(HttpStatus.FORBIDDEN)); + } } diff --git a/devGuide/README.md b/devGuide/README.md index 5c8f60577b..5e8486f013 100644 --- a/devGuide/README.md +++ b/devGuide/README.md @@ -9,6 +9,7 @@ This directory contains all development-related documentation for Stirling PDF. - **[Taskfile.yml](../Taskfile.yml)** - Unified task runner for all build/dev/test/lint commands - **[EXCEPTION_HANDLING_GUIDE.md](./EXCEPTION_HANDLING_GUIDE.md)** - Exception handling patterns and i18n best practices - **[HowToAddNewLanguage.md](./HowToAddNewLanguage.md)** - Internationalization and translation guide +- **[STORAGE_ENCRYPTION_AT_REST.md](./STORAGE_ENCRYPTION_AT_REST.md)** - Encryption at rest for stored files: key setup, migration, revocation, rotation ### Features & Documentation - **[AGENTS.md](./AGENTS.md)** - Agent-based functionality documentation diff --git a/devGuide/STORAGE_ENCRYPTION_AT_REST.md b/devGuide/STORAGE_ENCRYPTION_AT_REST.md new file mode 100644 index 0000000000..6ec8d047aa --- /dev/null +++ b/devGuide/STORAGE_ENCRYPTION_AT_REST.md @@ -0,0 +1,175 @@ +# Storage Encryption at Rest + +Encrypts files stored by Stirling (My Files, workflow files) so the bytes on disk, in the database, +or in S3 are unreadable without the master key. Requires a Pro or Enterprise licence to enable. + +> **Back up the master key.** Losing it makes every encrypted stored file permanently +> unrecoverable. There is no recovery path by design — that is what makes the encryption +> meaningful. + +> **Audit trail requires Enterprise.** Encryption itself works on Pro, but the audit events +> below (encrypt, decrypt, revocation, plaintext export, migration) are only recorded on an +> Enterprise licence — the audit subsystem is Enterprise-gated platform-wide. On Pro the files +> are encrypted exactly the same way, but there is no access trail, which matters if you are +> enabling this to satisfy an audit-logging requirement (HIPAA, CMMC). A warning is logged at +> startup when encryption is enabled without an Enterprise licence. + +## How it works + +Envelope encryption, three levels: + +| Level | What it is | Where it lives | +|---|---|---| +| Master key | Wraps the scope keys | Config property, env var, or `configs/file-encryption.key` | +| Scope key (KEK) | One per team; wraps each file's data key | `file_encryption_keys` table, master-key-wrapped | +| Data key (DEK) | One per stored blob; encrypts the bytes | Inside the blob's own header, scope-key-wrapped | + +Each blob is self-describing: an `SPDFEAR1` header carries the format version, the scope key's id, +the plaintext length, and the wrapped data key, followed by AES-256-GCM streaming ciphertext +(1 MiB segments). The header is bound as associated data to both the key wrap and the payload, so a +header cannot be transplanted between blobs. + +Consequences of that design worth knowing: + +- Blobs without the magic prefix are treated as plaintext and passed through, so enabling the + feature needs no migration and old files keep working. +- Because the key id is pinned per blob, moving a user between teams never breaks their existing + files. +- Plaintext sizes are what get recorded in the database, so quotas and `Content-Length` are + unaffected (ciphertext on disk is ~96 bytes + 16 bytes/MiB larger). +- Presigned S3 download URLs are suppressed once encrypted content can exist — a presigned GET + would hand raw ciphertext to the browser — so those downloads stream through the application. + +## Enabling it + +```yaml +storage: + encryption: + enabled: true +``` + +The master key is resolved in this order: + +1. `stirling.security.fileEncryptionKey` property +2. `STIRLING_FILE_ENCRYPTION_KEY` environment variable +3. an auto-generated `configs/file-encryption.key` (owner-only permissions) + +Generate a key with: + +```bash +openssl rand -base64 32 +``` + +It must decode to exactly 32 bytes; anything else fails at startup rather than silently +downgrading the cipher. The startup log prints a fingerprint (a SHA-256 prefix, never the key) so +you can verify a backup matches the live key. + +**Cluster mode** (`cluster.enabled=true`) requires the key to be set explicitly and identically on +every node; the auto-generated file is refused, because a node-local key would make files written +elsewhere unreadable. + +### Turning it off + +Disabling only stops encrypting *new* writes. Existing encrypted files stay readable as long as the +key material is present — the decrypt path is always active and is never licence-gated, so a lapsed +licence cannot lock you out of your own data. + +## Encrypting files that already exist + +Enabling the flag does not touch the existing plaintext backlog. To convert it: + +```bash +curl -X POST http://localhost:8080/api/v1/admin/storage-encryption/migrate +curl http://localhost:8080/api/v1/admin/storage-encryption/migrate/status +``` + +The job is throttled, resumable, and safe to re-run: for each file it writes the encrypted copy +under a new storage key, swaps the database row only if nothing else changed it, and deletes the old +blob last. If a user replaces a file mid-migration their copy wins and the job skips it. Progress is +in-memory, so a restart mid-run loses the counters and `migrate/status` reports `IDLE` again — just +start it again; already-encrypted files are skipped. There is currently no way to cancel a run, and +on a cluster the guard is per-node, so trigger the migration on one node only. + +## Revoking access (kill switch) + +Disabling a scope key makes every file already stored under it fail closed with `403` until it is +re-enabled: + +```bash +curl -X POST http://localhost:8080/api/v1/admin/storage-encryption/keys/{keyId}/disable +curl -X POST http://localhost:8080/api/v1/admin/storage-encryption/keys/{keyId}/enable +``` + +This revokes access to existing content; it does **not** stop the scope from storing new files. The +next upload finds no active key for the scope and mints one, so the team keeps working while its +history stays sealed. To stop new writes as well, turn encryption off (or take the scope's access +away at the application level) — the kill switch is aimed at stored bytes. + +Because of that, re-enabling is status-aware: the key returns to `ACTIVE` if its scope has no other +active key, and to `RETIRED` if one was minted while it was revoked. Both statuses decrypt existing +content; only `ACTIVE` wraps new writes, so a scope never ends up with two keys competing for new +uploads. The `enable` response reports which status was applied. + +This is reversible: the key material stays in the database and nothing is destroyed. No API path +deletes key material. On a cluster, other nodes pick the change up within their 60-second key-cache +window. + +## Rotating the master key + +Rotation only re-wraps the small `file_encryption_keys` table — file contents are never rewritten. + +1. Set the new key as `stirling.security.fileEncryptionKey`. +2. Keep the outgoing key in `stirling.security.fileEncryptionKeyPrevious`. +3. Bump `stirling.security.fileEncryptionKeyVersion`. +4. Restart. Startup warns about rows still wrapped by the previous key. On a cluster, wait until + **every** node carries both keys — a node still holding only the outgoing key cannot read a + re-wrapped row, so rotating mid-deploy makes the lagging nodes fail on those scopes until they + catch up. +5. `POST /api/v1/admin/storage-encryption/master/rotate`. +6. Confirm the response's `rewrapped` count and that `/status` shows every key row at the new + `masterKeyVersion`. +7. Remove `fileEncryptionKeyPrevious` and restart. + +**Do not skip step 6.** Until a row is re-wrapped it is still readable only with the outgoing key, so +removing that key while rows remain behind would seal the files under them. Startup verifies every +key row against the configured keys and refuses to start if any cannot be unwrapped, naming the count +and the first affected scope — so this shows up as a failed deploy, recoverable by putting the old key +back, rather than as unreadable files discovered later. Keep the outgoing key archived until a +restart has succeeded without it. + +Key material is never accepted over HTTP; the endpoint only performs the re-wrap step. + +## Auditing + +**Requires an Enterprise licence** (see the note at the top): on Pro these events are silently +dropped by the audit subsystem, and a warning is logged at startup. + +Encrypt, decrypt, denied-decrypt, key lifecycle, rotation, and migration events are written to the +audit trail, along with a `plaintextExport` marker whenever a plaintext copy of encrypted content is +served. Per-read decrypt events can be noisy on busy instances and can be turned off with +`storage.encryption.auditReads: false`; denials and key lifecycle events are always recorded. + +Two semantics worth knowing when reading the trail: + +- A `decrypt` event means a decryption was *authorised and opened*, not that bytes were read to + completion — a load that is discarded still records one, and a re-read of the same open resource + (e.g. an HTTP range request) does not record a second. +- `plaintextExport` is currently emitted for stored-file and share-link downloads. Workflow-file + downloads are not yet marked. + +## Status and backup verification + +```bash +curl http://localhost:8080/api/v1/admin/storage-encryption/status +``` + +Reports whether writes are encrypted, the master-key fingerprint, encrypted vs plaintext file +counts, and every key row with its status history. All endpoints under +`/api/v1/admin/storage-encryption` require an admin account. + +## What this protects against + +Stolen disks, database dumps, exposed object-storage buckets, decommissioned media, and platform +users who are not authorised for a file. It is not a defence against an attacker who already has +root on a running instance — at that point the key is in memory. No storage-level encryption product +claims otherwise.