Exclude Python virtualenvs from the Docker build context (#7658)

# Description of Changes

Stirling engine docker slimming

Exclude Python virtualenvs from the Docker build context
Drop unused provider SDKs from the engine dependency set
Retry the SQLite WAL switch when workers race on startup
Build the engine image in two stages and run it unprivileged
Swap voyage SDK for api call removing 200MB bloat
Bundle the AI engine in the fat image
Publish the AI engine as a standalone image


886MB to 295MB in docker file

And Docker fat is only 230MB bigger after adding (since it already has
python and some deps)

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.

engine/Dockerfile

@@ -1,35 +1,43 @@
 # syntax=docker/dockerfile:1.5
-FROM ghcr.io/astral-sh/uv:python3.13-bookworm-slim@sha256:531f855bda2c73cd6ef67d56b733b357cea384185b3022bd09f05e002cd144ca
 
-ARG TASK_VERSION=3.52.0
-RUN apt-get update \
-    && apt-get install -y --no-install-recommends curl ca-certificates \
-    && ARCH=$(dpkg --print-architecture) \
-    && curl -fsSL "https://github.com/go-task/task/releases/download/v${TASK_VERSION}/task_${TASK_VERSION}_linux_${ARCH}.deb" -o /tmp/task.deb \
-    && dpkg -i /tmp/task.deb \
-    && rm /tmp/task.deb \
-    && rm -rf /var/lib/apt/lists/*
+# uv resolves the venv here so its ~52MB binary stays out of the runtime image.
+FROM ghcr.io/astral-sh/uv:python3.13-bookworm-slim@sha256:531f855bda2c73cd6ef67d56b733b357cea384185b3022bd09f05e002cd144ca AS builder
 
-# Source under /app/engine/ to match root Taskfile's `includes.engine.dir: engine`.
 WORKDIR /app/engine
-COPY engine/pyproject.toml engine/uv.lock engine/.env ./
-COPY engine/scripts/ ./scripts/
+COPY engine/pyproject.toml engine/uv.lock ./
 RUN --mount=type=cache,target=/root/.cache/uv \
-    uv sync --frozen --no-dev --group engine
+    uv sync --frozen --no-dev --no-install-project --group engine
 
-COPY engine/src/ ./src/
-WORKDIR /app
-COPY Taskfile.yml ./
-COPY .taskfiles/ ./.taskfiles/
+FROM python:3.13-slim-bookworm@sha256:00faa2debb87529f9f0764e9491d8ba400a3678976616c3bd7cb193745ac20d1 AS runtime
+
+# Created before the COPYs so they land owned; a later chown -R duplicates the venv layer.
+RUN set -eux; \
+    groupadd --system --gid 1000 stirling; \
+    useradd --system --uid 1000 --gid 1000 --home /app/engine stirling; \
+    mkdir -p /app/engine/data; \
+    chown stirling:stirling /app/engine /app/engine/data
+
+WORKDIR /app/engine
+COPY --from=builder --chown=stirling:stirling /app/engine/.venv ./.venv
+# settings.py resolves ENGINE_ROOT to /app/engine, so .env must sit here.
+COPY --chown=stirling:stirling engine/.env ./
+COPY --chown=stirling:stirling engine/src/ ./src/
 
 ENV PATH="/app/engine/.venv/bin:$PATH"
 ENV PYTHONUNBUFFERED=1
 ENV STIRLING_ENGINE_WORKERS=4
-# Container runs on a fixed port; skip the host-only free-port probe (its script
-# is not shipped in the image). engine:run honours these.
-ENV ENGINE_PORT_PROBE=false
 ENV STIRLING_ENGINE_PORT=5001
+# Fail closed: without a secret the document routes trust caller-supplied X-User-Id.
+# Set STIRLING_ENGINE_SHARED_SECRET (the backend sends it as X-Engine-Auth), or false to opt out.
+ENV STIRLING_ENGINE_REQUIRE_AUTH=true
+
+# `stirling` resolves from the working directory.
+WORKDIR /app/engine/src
+USER stirling
 
 EXPOSE 5001
 
-CMD ["task", "engine:run"]
+HEALTHCHECK --interval=30s --timeout=5s --start-period=40s --retries=3 \
+  CMD ["python", "-c", "import os,sys,urllib.request; sys.exit(0 if urllib.request.urlopen('http://127.0.0.1:%s/health' % os.environ.get('STIRLING_ENGINE_PORT','5001'), timeout=4).status==200 else 1)"]
+
+CMD ["sh", "-c", "exec uvicorn stirling.api.app:app --host 0.0.0.0 --port ${STIRLING_ENGINE_PORT:-5001} --workers ${STIRLING_ENGINE_WORKERS:-4}"]