Commit Graph
12 Commits
Author SHA1 Message Date
Anthony Stirling ddee690c58 Exclude Python virtualenvs from the Docker build context (#7658)
# Description of Changes

Stirling engine docker slimming

Exclude Python virtualenvs from the Docker build context
Drop unused provider SDKs from the engine dependency set
Retry the SQLite WAL switch when workers race on startup
Build the engine image in two stages and run it unprivileged
Swap voyage SDK for api call removing 200MB bloat
Bundle the AI engine in the fat image
Publish the AI engine as a standalone image


886MB to 295MB in docker file

And Docker fat is only 230MB bigger after adding (since it already has
python and some deps)

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-08-24 16:23:55 +00:00
brios c19926456c build(deps): update built from source dep versions in Dockerfiles (#7412)
# Description of Changes

PR updates built from source dep versions in Dockerfiles

Changes:

* Updated `CALIBRE_VERSION` from 9.4.0 to 9.13.0 in
`docker/base/Dockerfile`.
* Updated `GS_VERSION` (Ghostscript) from 10.06.0 to 10.07.1 in
`docker/base/Dockerfile`.
* Updated `IM_VERSION` (ImageMagick) from 7.1.2-13 to 7.1.2-29 in
`docker/base/Dockerfile`.
* Updated `QPDF_VERSION` is already at 12.3.2, no change.
* Updated `UNOSERVER_VERSION` from 3.6 to 3.7 in both
`docker/base/Dockerfile` and `docker/unoserver/Dockerfile` to align with
the client version and avoid wire mismatches.
* Updated `TASK_VERSION` from 3.49.1 to 3.52.0 in
`docker/embedded/Dockerfile`, `docker/embedded/Dockerfile.fat`,
`docker/embedded/Dockerfile.ultra-lite`, and `engine/Dockerfile`.
<!--
Please provide a summary of the changes, including:

- What was changed
- Why the change was made
- Any challenges encountered

Closes #(issue_number)
-->

---

## Checklist

### General

- [X] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [X] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [X] I have performed a self-review of my own code
- [X] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [X] I have run `task check` to verify linters, typechecks, and tests
pass
- [X] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-08-20 14:41:07 +00:00
05eb74022a chore(ci): migrate Python tooling to uv and standardize workflow execution (#7386)
# Description of Changes

This PR modernizes the project's Python tooling across GitHub Actions by
migrating CI workflows from pip-based dependency management to `uv` and
aligning Python execution with the engine project's managed environment.

### What was changed

- Replaced `actions/setup-python` and ad-hoc `pip install` steps with
`astral-sh/setup-uv` across CI workflows.
- Configured shared `uv` dependency caching using
`engine/pyproject.toml` and `engine/uv.lock`.
- Updated Python script execution to use `uv run --project engine
--locked` for a consistent runtime environment.
- Replaced package installation steps with `uv sync` for the required
dependency groups (e.g. `tools` and `cucumber`).
- Added Docker image build validation for both production and
development AI engine images.
- Updated workflow cache configuration and Docker build context where
required.
- Removed obsolete Python requirements files that are no longer needed
after the migration.
- Applied minor Python code modernizations, including import cleanup,
modern built-in generic type annotations (`list[...]`, `tuple[...]`,
`float | None`), and small style improvements.
- Removed unnecessary Python formatter/linter extensions from the
development container configuration.

### Why the change was made

- Standardize Python dependency management across the repository.
- Reduce duplicated dependency installation logic in CI.
- Improve workflow performance through shared dependency caching.
- Ensure all Python utilities execute against the same locked dependency
set managed by the engine project.
- Simplify long-term maintenance by eliminating legacy requirements
files and pip-specific workflow steps.


---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.

---------

Signed-off-by: Carsten Drewes <c.drewes@stud.uni-hannover.de>
Co-authored-by: albanobattistella <34811668+albanobattistella@users.noreply.github.com>
Co-authored-by: kastenherri <116314318+kastenherri@users.noreply.github.com>
Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: James Brunton <jbrunton96@gmail.com>
2026-08-11 08:09:21 +00:00
Anthony Stirling d48017a5b5 Skip engine free-port probe in containers (fixed port) 2026-06-11 20:18:54 +01:00
James Brunton d52c7ced7c Improvements to Stirling Engine to prepare for SaaS release (#6603)
# Description of Changes
- Use pool for postgres connections
- Add ability to require user ID to be set on API calls to the engine
- Add process-wide concurrency cap on AI access (in addition to existing
user caps)
- Allow number of workers (threads) to be specified for stirling engine
- Update env var names to reflect that the DB is not just for RAG
2026-06-11 16:31:35 +01:00
Anthony Stirling 1d5ce8a1d2 chore: shorten verbose block comments across SaaS branch 2026-06-08 18:38:00 +01:00
Anthony Stirling d9651f7065 fix(engine): match Dockerfile layout to root Taskfile dir: engine 2026-06-08 18:02:02 +01:00
Ludy e087b54cf0 build(docker): pin base container images to immutable digests (#6173) 2026-04-23 13:31:21 +01:00
James Brunton 3b2afe0deb Change engine/.env to be committed and have .env.local override (#6150)
# Description of Changes
We keep adding stuff to `engine/config/.env.example` and have to
manually update `.env` because of it, which is really clunky, especially
when working on multiple worktrees at once. This PR changes it so that
we just have a committed `.env` file and have an `.env.local` override
to put the actual private keys into, which should make it a bit easier
to manage.

> [!warning]
>
> After this goes in, be very careful for a little while not to
accidentally commit any keys that you've got inside your `.env` file!
2026-04-21 16:18:25 +01:00
EthanHealy01andJames Brunton 089e448cf4 allow deploypr:prototypes comment to spin up the prototypes build (#6144)
Co-authored-by: James Brunton <jbrunton96@gmail.com>
2026-04-20 18:58:33 +01:00
James Brunton e10c5f6283 Redesign Python AI engine (#5991)
# Description of Changes
Redesign the Python AI engine to be properly agentic and make use of
`pydantic-ai` instead of `langchain` for correctness and ergonomics.
This should be a good foundation for us to build our AI engine on going
forwards.
2026-03-26 10:35:47 +00:00
James Brunton c58a6092ec Add SaaS AI engine (#5907) 2026-03-16 11:01:50 +00:00