Stirling-Tools/Stirling-PDF
1
0
Fork 0
mirror of https://github.com/Stirling-Tools/Stirling-PDF.git synced 2026-09-02 21:03:34 +03:00
Code Issues Packages Projects Releases Wiki Activity

chore(ci): migrate Python tooling to uv and standardize workflow execution (#7386)

Browse Source 
# Description of Changes

This PR modernizes the project's Python tooling across GitHub Actions by
migrating CI workflows from pip-based dependency management to `uv` and
aligning Python execution with the engine project's managed environment.

### What was changed

- Replaced `actions/setup-python` and ad-hoc `pip install` steps with
`astral-sh/setup-uv` across CI workflows.
- Configured shared `uv` dependency caching using
`engine/pyproject.toml` and `engine/uv.lock`.
- Updated Python script execution to use `uv run --project engine
--locked` for a consistent runtime environment.
- Replaced package installation steps with `uv sync` for the required
dependency groups (e.g. `tools` and `cucumber`).
- Added Docker image build validation for both production and
development AI engine images.
- Updated workflow cache configuration and Docker build context where
required.
- Removed obsolete Python requirements files that are no longer needed
after the migration.
- Applied minor Python code modernizations, including import cleanup,
modern built-in generic type annotations (`list[...]`, `tuple[...]`,
`float | None`), and small style improvements.
- Removed unnecessary Python formatter/linter extensions from the
development container configuration.

### Why the change was made

- Standardize Python dependency management across the repository.
- Reduce duplicated dependency installation logic in CI.
- Improve workflow performance through shared dependency caching.
- Ensure all Python utilities execute against the same locked dependency
set managed by the engine project.
- Simplify long-term maintenance by eliminating legacy requirements
files and pip-specific workflow steps.


---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.

---------

Signed-off-by: Carsten Drewes <c.drewes@stud.uni-hannover.de>
Co-authored-by: albanobattistella <34811668+albanobattistella@users.noreply.github.com>
Co-authored-by: kastenherri <116314318+kastenherri@users.noreply.github.com>
Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: James Brunton <jbrunton96@gmail.com>
This commit is contained in:
Ludy
2026-08-11 08:09:21 +00:00
committed by GitHub
co-authored by albanobattistella kastenherri Anthony Stirling Copilot James Brunton
parent bbaff8d6c4
commit 05eb74022a
64 changed files with 1684 additions and 3258 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.devcontainer/devcontainer.json
-2
View File
@@ -119,8 +119,6 @@
"extensions": [ "extensions": [
"elagil.pre-commit-helper", // Support for pre-commit hooks to enforce code quality "elagil.pre-commit-helper", // Support for pre-commit hooks to enforce code quality
"josevseb.google-java-format-for-vs-code", // Google Java code formatter to follow the Google Java Style Guide "josevseb.google-java-format-for-vs-code", // Google Java code formatter to follow the Google Java Style Guide
"ms-python.black-formatter", // Python code formatter using Black
"ms-python.flake8", // Flake8 linter for Python to enforce code quality
"ms-python.python", // Official Microsoft Python extension with IntelliSense, debugging, and Jupyter support "ms-python.python", // Official Microsoft Python extension with IntelliSense, debugging, and Jupyter support
"ms-vscode-remote.vscode-remote-extensionpack", // Remote Development Pack for SSH, WSL, and Containers "ms-vscode-remote.vscode-remote-extensionpack", // Remote Development Pack for SSH, WSL, and Containers
// "Oracle.oracle-java", // Oracle Java extension with additional features for Java development // "Oracle.oracle-java", // Oracle Java extension with additional features for Java development
.github/scripts/check_language_toml.py
+9 -28
View File
@@ -19,9 +19,9 @@ import argparse
import glob import glob
import os import os
import re import re
import tomllib # Python 3.11+ (stdlib)
from pathlib import Path from pathlib import Path
import tomllib # Python 3.11+ (stdlib)
import tomli_w # For writing TOML files import tomli_w # For writing TOML files
@@ -133,11 +133,7 @@ def update_missing_keys(reference_file, file_list, branch=""):
file_path = Path(file_path) file_path = Path(file_path)
language_dir = file_path.parent.name language_dir = file_path.parent.name
reference_lang_dir = reference_file.parent.name reference_lang_dir = reference_file.parent.name
if ( if language_dir == reference_lang_dir or file_path.suffix != ".toml" or file_path.parents[1].name != "locales":
language_dir == reference_lang_dir
or file_path.suffix != ".toml"
or file_path.parents[1].name != "locales"
):
print(f"Skipping file: {file_path}") print(f"Skipping file: {file_path}")
continue continue
@@ -198,9 +194,7 @@ def check_for_differences(reference_file, file_list, branch, actor):
# Verify that file is within the expected directory # Verify that file is within the expected directory
if not absolute_path.is_relative_to(base_dir): if not absolute_path.is_relative_to(base_dir):
has_differences = True has_differences = True
report.append( report.append(f"\n⚠️ Unsafe file found: `{locale_dir}/{basename_current_file}`\n\n---\n")
f"\n⚠️ Unsafe file found: `{locale_dir}/{basename_current_file}`\n\n---\n"
)
continue continue
# Verify file size before processing # Verify file size before processing
@@ -214,10 +208,7 @@ def check_for_differences(reference_file, file_list, branch, actor):
if basename_current_file == basename_reference_file and locale_dir == "en-US": if basename_current_file == basename_reference_file and locale_dir == "en-US":
continue continue
if ( if file_normpath.suffix != ".toml" or basename_current_file != "translation.toml":
file_normpath.suffix != ".toml"
or basename_current_file != "translation.toml"
):
continue continue
only_reference_file = False only_reference_file = False
@@ -261,9 +252,7 @@ def check_for_differences(reference_file, file_list, branch, actor):
) )
report.append("") report.append("")
report.append(" Use the following command to remove them:") report.append(" Use the following command to remove them:")
report.append( report.append(f" `python scripts/translations/translation_merger.py {locale_dir} remove-unused`")
f" `python scripts/translations/translation_merger.py {locale_dir} remove-unused`"
)
report.append("") report.append("")
if extra_keys_list: if extra_keys_list:
report.append( report.append(
@@ -271,9 +260,7 @@ def check_for_differences(reference_file, file_list, branch, actor):
) )
report.append("") report.append("")
report.append(" Use the following command to add them:") report.append(" Use the following command to add them:")
report.append( report.append(f" `python scripts/translations/translation_merger.py {locale_dir} add-missing`")
f" `python scripts/translations/translation_merger.py {locale_dir} add-missing`"
)
report.append("") report.append("")
if missing_keys_list or extra_keys_list: if missing_keys_list or extra_keys_list:
@@ -288,9 +275,7 @@ def check_for_differences(reference_file, file_list, branch, actor):
output = "\n".join( output = "\n".join(
[ [
f" - `{key}`: first at {first}, duplicate at `{duplicate}`" f" - `{key}`: first at {first}, duplicate at `{duplicate}`"
for key, first, duplicate in find_duplicate_keys( for key, first, duplicate in find_duplicate_keys(branch_path / file_normpath)
branch_path / file_normpath
)
] ]
) )
report.append("3. **Test Status:** ❌ **_Failed_**") report.append("3. **Test Status:** ❌ **_Failed_**")
@@ -313,18 +298,14 @@ def check_for_differences(reference_file, file_list, branch, actor):
else: else:
report.append("## ✅ Overall Check Status: **_Success_**") report.append("## ✅ Overall Check Status: **_Success_**")
report.append("") report.append("")
report.append( report.append(f"Thanks @{actor} for your help in keeping the translations up to date.")
f"Thanks @{actor} for your help in keeping the translations up to date."
)
if not only_reference_file: if not only_reference_file:
print("\n".join(report)) print("\n".join(report))
if __name__ == "__main__": if __name__ == "__main__":
parser = argparse.ArgumentParser( parser = argparse.ArgumentParser(description="Find missing keys in TOML translation files")
description="Find missing keys in TOML translation files"
)
parser.add_argument( parser.add_argument(
"--actor", "--actor",
required=False, required=False,
.github/scripts/requirements_dev.in
-9
View File
@@ -1,9 +0,0 @@
pip
setuptools
WeasyPrint
pdf2image
pillow
unoserver
opencv-python-headless
pre-commit
brotli @ git+https://github.com/google/brotli.git@028fb5a23661f123017c060daa546b55cf4bde29
.github/scripts/requirements_dev.txt
-528
View File
@@ -1,528 +0,0 @@
#
# This file is autogenerated by pip-compile with Python 3.13
# by the following command:
#
# pip-compile --allow-unsafe --generate-hashes --output-file='.github\scripts\requirements_dev.txt' --strip-extras '.github\scripts\requirements_dev.in'
#
# WARNING: pip install will require the following package to be hashed.
# Consider using a hashable URL like https://github.com/jazzband/pip-tools/archive/SOMECOMMIT.zip
brotli @ git+https://github.com/google/brotli.git@028fb5a23661f123017c060daa546b55cf4bde29
# via
# -r .github/scripts/requirements_dev.in
# fonttools
cffi==2.1.0 \
--hash=sha256:02cb7ff33ded4f1532476731f89ede53e2e488a8e6205515a82144246ffa7dcc \
--hash=sha256:03e9810d18c646077e501f661b682fbf5dee4676048527ca3cffe66faa9960dd \
--hash=sha256:0520e1f4c35f44e209cbbb421b67eec42e6a157f59444dfb6058874ff3610e5d \
--hash=sha256:0582a58f3051372229ca8e7f5f589f9e5632678208d8636fea3676711fdf7fe5 \
--hash=sha256:0611e7ebf90573a535ebdc33ae9da222d037853983e13359f580fab781ca017f \
--hash=sha256:0a42c688d19fca6e095a53c6a6e2295a5b050a8b289f109adab02a9e61a25de6 \
--hash=sha256:0a96b74cda968eebbad56d973efe5098974f0a9fb323865bf99ea1fd24e3e64c \
--hash=sha256:10537b1df4967ca26d21e5072d7d54188354483b91dc75058968d3f0cf13fbda \
--hash=sha256:11b3fb55f4f8ad92274ed26705f65d8f91457de71f5380061eb6d125a768fecd \
--hash=sha256:15faec4adfff450819f3aee0e2e02c812de6edb88203aa58807955db2003472a \
--hash=sha256:164bff1657b2a74f0b6d54e11c9b375bc97b931f2ca9c43fcf875838da1570dd \
--hash=sha256:1854b724d00f6654c742097d5387569021be12d3a0f770eae1df8f8acfcc6acd \
--hash=sha256:19c54ac121cad98450b4896fa9a43ee0180d57bc4bc911a33db6cab1efab6cd3 \
--hash=sha256:1b96bfe2c4bd825681b7d311ad6d9b7280a091f43e8f63da5729638083cd3bfb \
--hash=sha256:1e9f50d192a3e525b15a75ab5114e442d83d657b7ec29182a991bc9a88fd3a66 \
--hash=sha256:1ff3456eab0d889592d1936d6125bbfbc7ae4d3354a700f8bd80450a66445d4d \
--hash=sha256:2282cd5e38aa8accd03e99d1256af8411c84cdbee6a89d841b563fdbd1f3e50f \
--hash=sha256:276f20fffd7b396e12516ba8edf9509210ac248cbbc5acbc39cd512f9f59ebe6 \
--hash=sha256:2b71d409cccee78310ab5dec549aed052aaea483346e282c7b02362596e01bb0 \
--hash=sha256:2e9dabb9abcb7ad15938c7196ad5c1718a4e6d33cc79b4c0209bdb64c4a54a5c \
--hash=sha256:30b65779d598c370374fefabf138d456fd6f3216bfa7bedfab1ba82025b0cd93 \
--hash=sha256:33eb1ad83ebe8f313e0df035c406227d55a79456704a863fad9842136af5ad7d \
--hash=sha256:35aaea0c7ee0e58a5cd8c2fd1a48fdf7ece0d2699b7ecdda08194e9ce5dd9b3d \
--hash=sha256:3681e031db29958a7502f5c0c9d6bbc4c36cb20f7b104086fa642d1799631ff8 \
--hash=sha256:379de10ce1ba048b1448599d1b37b24caee16309d1ac98d3982fc997f768700b \
--hash=sha256:37f525a7e7e50c017fdebe58b787be310ad59357ae43a053943a6e1a6c526001 \
--hash=sha256:3b926723c13eba9f81d2ef3820d63aeceec3b2d4639906047bf675cb8a7a500d \
--hash=sha256:3d7f118b5adbfdfead90c25822690b02bc8074fba949bb7858bec4ebd55adb43 \
--hash=sha256:46b1c8db8f6122420f32d02fffb924c2fe9bc772d228c7c711748fff56aabb2b \
--hash=sha256:47ff3a8bfd8cb9da1af7524b965127095055654c177fcfc7578debcb015eecd0 \
--hash=sha256:4d433a51f1870e43a13b6732f92aaf540ff77c2015097c78556f75a2d6c030e0 \
--hash=sha256:4f26194e3d95e06501b942642855aed4f953d55e95d7d01b7c4483db3ecff458 \
--hash=sha256:510aeeeac94811b138077451da1fb18b308a5feab47dd2b603af55804155e1c8 \
--hash=sha256:5972433ad71a9e46516584ef60a0fda12d9dc459938d1539c3ddecf9bdc1368d \
--hash=sha256:5ecbd0499275d57506d397eebe1981cee87b47fcd9ef5c22cab7ed7644a39a94 \
--hash=sha256:6274dcb2d15cef48daa73ed1be5a40d501d74dccd0cd6db364776d12cb6ba022 \
--hash=sha256:63960549e4f8dc41e31accb97b975abaecfc44c03e396c093a6436763c2ea7db \
--hash=sha256:64c753a0f87a256020004f37a1c8c02c480e725f910f0b2a0f3f07debd1b2479 \
--hash=sha256:6af371f3767faeffc6ac1ef57cdfd25844403e9d3f476c5537caee499de96376 \
--hash=sha256:6ca4919c6e4f89aa99c42510b42cf54596892c00b3f9077f6bdd1505e24b9c8d \
--hash=sha256:6d194185eabd279f1c05ebe3504265ddfc5ad2b58d0714f7db9f01da592e9eb6 \
--hash=sha256:702c436735fbe99d59ada02a1f65cfc0d31c0ee8b7290912f8fbc5cd1e4b16c3 \
--hash=sha256:716ff8ec22f20b4d988b12884086bcef0fc99737043e503f7a3935a6be99b1ea \
--hash=sha256:762f99479dcb369f60ab9017ad4ab97a36a1dd7c1ee5a3b15db0f4b8659120cd \
--hash=sha256:7762faa47e8ff7eb80bd261d9a7d8eea2d8baa69de5e95b70c1f338bbe712f02 \
--hash=sha256:78474632761faa0fb96f30b1c928c84ebcf68713cbb80d15bab09dfe61640fde \
--hash=sha256:799416bae98336e400981ff6e532d67d5c709cfb30afb79865a1315f94b0e224 \
--hash=sha256:7d034dcffa09e9a46c93fa3a3be402096cb5354ac6e41ab8e5cc9cd8b642ad76 \
--hash=sha256:7d28dff1db6764108bc30788d85d61c876beff416d9a49cb9dd7c5a9f34f5804 \
--hash=sha256:7d3538f9c0e50670f4deb93dbb696576e60590369cae2faf7de681e597a8a1f1 \
--hash=sha256:7d5980a3433d4b71a5e120f9dd551403d7824e31e2e67124fe2769c404c06913 \
--hash=sha256:7ea6b3e2c4250ff1de21c630fe72d0f63eb95c2c32ffbf64a358cf4a8836d714 \
--hash=sha256:86cf8755a791f72c85dc287128cc62d4f24d392e3f1e15837245623f4a33cccc \
--hash=sha256:88023dfe18799507b73f1dbb0d14326a17465de1bc9c9c7655c22845e9ddc3a2 \
--hash=sha256:89095c1968b4ba8285840e131bf2891b09ae137fe2146905acae0354fbce1b5e \
--hash=sha256:8d35c139744adb3e727cd51b1a18324bbe44b8bd41bf8322bca4d41289f48eda \
--hash=sha256:8e74a6135550c4748af665b1b1118b6aab33b1fc6a16f9aff630af107c3b4512 \
--hash=sha256:8f9ec95b8a043d3dfbc74d9abc6f7baf524dd27a8dc160b0a32ff9cdab650c28 \
--hash=sha256:90bec57cf82089383bd06a605b3eb8daebf7e5a668520beaf6e327a83a947699 \
--hash=sha256:95f2954c2c9473d892eca6e0409f3568b37ab62a8eedb122461f73cc273476e3 \
--hash=sha256:961be50688f7fba2fa65f63712d3b9b341a22311f5253460ce933f52f0de1c8c \
--hash=sha256:98fff996e983a36d3aa2eca83af40c5821202e7e6f32d13ae94e3d2286f10cfe \
--hash=sha256:9b8f0f26ca4e7513c534d351eca551947d053fac438f2a04ac96d882909b0d3a \
--hash=sha256:9d72af0cf10a76a600a9690078fe31c63b9588c8e86bf9fd353f713c84b5db0f \
--hash=sha256:9d8272c0e483b024e1b9ad029821470ed8ec65631dbd90217469da0e7cd89f1c \
--hash=sha256:a016194dbe13d14ee9556e734b772d8d67b947092b268d757fd4290e3ba2dfc2 \
--hash=sha256:a5781494d4d400a3f47f8f1da94b324f6e6b440a53387774002890a2a2f4b50f \
--hash=sha256:a95b05f9baf29b91171b3a8bd2020b028835243e7b0ff6bb23e2a3c228518b1b \
--hash=sha256:aa7a1b53a2a4452ada2d1b5dade9960b2522f1e61293a811a077439e39029565 \
--hash=sha256:ac0f1a2d0cfa7eea3f2aaf006ab6e70e8feeb16b75d65b7e5939982ca2f11056 \
--hash=sha256:af5e2915d41fe6c961694d7bfdc8562942638200f3ce2765dfb8b745cf997629 \
--hash=sha256:b6422532152adf4e59b110cb2808cee7a033800952f5c036b4af047ee43199e7 \
--hash=sha256:b65f590ef2a44640f9a05dbb548a429b4ade77913ce683ac8b1480777658a6c0 \
--hash=sha256:ba00f661f8ba35d075c937174e27c2c421cec3942fd2e0ea3e66996757c0fdd9 \
--hash=sha256:bccbbb5ee76a61f9d99b5bf3846a51d7fca4b6a732fe46f89295610edaf41853 \
--hash=sha256:bf01d8c84cbea96b944c73b22182e6c7c432b3475632b8111dbfdc95ddad6e13 \
--hash=sha256:bf5c6cf48238b0eb4c086978c492ad1cbc22373fc5b2d7353b3a598ce6db887a \
--hash=sha256:c16914df9fb7f500e440e6875fa23ff5e0b31db01fa9c06af98d59a91f0dc2e4 \
--hash=sha256:c351efb95e832a853a29361675f33a7ce53de1a109cd73fd47af0712213aa4ce \
--hash=sha256:c4165821e131d6d4ca444347c2b694e2311bcfa3fe5a861cc72968f28867beac \
--hash=sha256:c5f5df567f6eb216de69be06ce55c8b714090fae02b18a3b40da8163b8c5fa9c \
--hash=sha256:c941bb58d5a6e1c3892d86e42927ed6c180302f07e6d395d08c416e594b98b46 \
--hash=sha256:c97f080ea627e2863524c5af3836e2270b5f5dfff1f104392b959f8df0c5d384 \
--hash=sha256:cb96698e3c7413d906ce83f8ffd245ec1bd94707541f299d0ce4d6b0193e982b \
--hash=sha256:cbb7640ce37159548d2147b5b8c241f962143d4c71231431820783f4dc78f210 \
--hash=sha256:cdf2448aab5f661c9315308ec8b93f4e8a1a67a3c733f8631067a2b67d5913dc \
--hash=sha256:d2117334c3af3bdcb9a88522b844a2bdb5efdc4f71c6c822df55486ae1c3347a \
--hash=sha256:d53d10f7da99ae46f7373b9150393e9c5eab9b224909982b43832668de4779f5 \
--hash=sha256:d9fafc5aa2e2a39aaf7f8cc0c1f044a9b07fca12e558dca53a3cc5c654ad67a7 \
--hash=sha256:db3eb7d46527159a878ec3460e9d40615bc25ba337d477db681aea6e4f05c5d2 \
--hash=sha256:dbf7c7a88e2bac086f06d14577332760bdeecc42bdec8ac4077f6260557d9326 \
--hash=sha256:df2b82571a1b30f58a87bf4e5a9e78d2b1eff6c6ce8fd3aa3757221f93f0863f \
--hash=sha256:df92f2aba50eb4d96718b68ef76f2e57a57b54f2fa62333496d16c6d585a85ca \
--hash=sha256:eb4e8997a49aa2c08a3e43c9045d224448b8941d88e7ac163c7d383e560cbf98 \
--hash=sha256:efc1cdd798b1aaf39b4610bba7aad28c9bea9b910f25c784ccf9ec1fa719d1f9 \
--hash=sha256:f146d154428a2523f9cc7936c02353c2459b8f6cf07d3cd1ee1c0a611109c5d5 \
--hash=sha256:f5bce581e6b8c235e566a14768a943b172ada3ed73537bb0c0be1edee312d4e7 \
--hash=sha256:f9912624a0c0b834b7520d7769b3644453aabc0a7e1c839da7359f050750e9bc \
--hash=sha256:fb62edb5bb52cca65fab91a63afa7561607120d26090a7e8fda6fb9f064726da \
--hash=sha256:ff067a8d8d880e7809e4ac88eb009bb848870115317b306666502ccad30b147f
# via weasyprint
cfgv==3.5.0 \
--hash=sha256:a8dc6b26ad22ff227d2634a65cb388215ce6cc96bbcc5cfde7641ae87e8dacc0 \
--hash=sha256:d5b1034354820651caa73ede66a6294d6e95c1b00acc5e9b098e917404669132
# via pre-commit
cssselect2==0.9.0 \
--hash=sha256:6a99e5f91f9a016a304dd929b0966ca464bcfda15177b6fb4a118fc0fb5d9563 \
--hash=sha256:759aa22c216326356f65e62e791d66160a0f9c91d1424e8d8adc5e74dddfc6fb
# via weasyprint
distlib==0.4.3 \
--hash=sha256:4b0ce306c966eb73bc3a7b6abad017c556dadd92c44701562cd528ac7fde4d5b \
--hash=sha256:f152097224a0ae24be5a0f6bae1b9359af82133bce63f98a95f86cae1aede9ed
# via virtualenv
filelock==3.30.0 \
--hash=sha256:1774e682dbe443bd60f9609162fc596e2c80dc84ffc2957068953406d0520090 \
--hash=sha256:40632998f0772e64183bb819f086a1b9def6be1090cf1dcb9d45f46806ef279b
# via
# python-discovery
# virtualenv
fonttools==4.63.0 \
--hash=sha256:032038247a96c1690f9f31e377c389383c902531b085aa4e4dabd6f57f870e69 \
--hash=sha256:063e08bd17bd5a90127a14123de0d6a952dbc847695fd98b63c043d58057f90c \
--hash=sha256:0c18358a155d75034911c5ee397a5b44cd19dd325dbb8b35fb60bf421d6a72ac \
--hash=sha256:0eac00b9118c3c2f87d272e45341871c5b3066baa3c86897fa634a7c3fb59096 \
--hash=sha256:1e874792a8212b44583ea02189d9e693906b2f78b261f372f95d6c563210ac1d \
--hash=sha256:22135da48a348785c5e2d5d2d9d6bec5ed44adacbaeb9db12d9493bf6c6bfa68 \
--hash=sha256:22693918177bd9ceabec4736d338045f357769416fc6b0b2508eefef75b08616 \
--hash=sha256:27fdc65af8da6f88b9c6121c47a464cbe359fcfff7ff6fc2d37a1f395d755b78 \
--hash=sha256:2b8ae05d9eacf6081414d759c0a352769ac28ce31280d6bb8e77b03f9e3c449f \
--hash=sha256:2c14b4fd138c4bafcca294765c547914e1aa431ae1ca94ab99d8db08c958bd3b \
--hash=sha256:308f957cdeaf8abe4e5f2f124902ef405448af92c90f80e302a3b771c2e6116b \
--hash=sha256:37dd23e621e3b0aef1baa70a303b80aaf38449632cfc8fd2a55fb285bbccfc02 \
--hash=sha256:445af2eab030a16b9171ea8bdda7ebf7d96bda2df88ee182a464252f6e05e20d \
--hash=sha256:51394295f1a51de8b5f30bdb1e1b9a4231536c7064ef5c6e211eec19fa36036f \
--hash=sha256:58dc6bb86a78d782f00f9190ca02c119cf5bbe2807536e361e18d42019f877d8 \
--hash=sha256:59ac449f8cca9b4ffa08d2e7bbadad87ce710d69d1eda5c3c1ce579baa987272 \
--hash=sha256:6b2248c5decb223562f7902ff6325077a073f608ee8e33e88ad88db734eb9f49 \
--hash=sha256:6d4741eb179121cab9eea4cb2393d24492373a260d7945006358c08cfbf45419 \
--hash=sha256:6db5140a60a5d731d21ec076745b40a310607731b0a565b50776393188649001 \
--hash=sha256:6e528da43bc3791085f8cb6141b1d13e459226790240340fcbb4625649238b03 \
--hash=sha256:796f27556dbe094c4824f75ca85267e4df776c79036c8441469a4df37038c196 \
--hash=sha256:79cdc9f567aec74a72918fd060283911406750cbc9fd28c1316023deb6ce31a9 \
--hash=sha256:7d76edbff9014094dbf03bd2d074709dfa6ec7aba13d838c937a2b33d2d6a86e \
--hash=sha256:7d782fac32985914c351556f68ac0855391572bcd87de50e05970d3cd4c96fc5 \
--hash=sha256:7dd683fef0663e9f0f45cf541d788d24caa3ec9db50796b588e1757d8b3bc007 \
--hash=sha256:85be818f5506e8a7753153def2c9550178f0ecae6a47b5e0e8dbb23f7cc90380 \
--hash=sha256:948428a275741f0b64b113c955425a953314f4b9ab9997f73a72c83e68e569c8 \
--hash=sha256:9ced0bd02ac751dd6319b0da88aaef24414e3b0dbc32bb4f24944821a3741a27 \
--hash=sha256:9e12f105d2b6342c559c298afb674006bb2893afc7102dcf8a1b55b0486b4e40 \
--hash=sha256:a8b33a82979e0a6a34ff435cc81317be1f95ec1ebb7a3a2d1c8a6a54f02ae44e \
--hash=sha256:a9faff9e0c1f76f9fd55899d2ce785832efebab37eb8ae13995853aef178bef0 \
--hash=sha256:af2fd1664d00a397d75f806985ddb36282091c2131a73a6485c23b4a34722263 \
--hash=sha256:afefc1ed0a59785a7fb06ea7e1678e849c193e1e387db783579bc7b3056fcfcb \
--hash=sha256:b1cd75a03ad8cb5bc40c90bfde68c0c47de423aa19e5c0f362b43520645eea94 \
--hash=sha256:ba04cb5891d4c0c21b6da95eda8d7b090021508a294fff33464fc7d241e0856b \
--hash=sha256:bf00f21eb5fb721dbaf73d1e9da6d02a1af7768f2ebcf9798be98beab8ba90f6 \
--hash=sha256:c0425b277a59cff3d80ca42162a8de360f318438a2ac83570842a678d826d579 \
--hash=sha256:c1aaa4b9c75798400ac043ce04d74e7830376c85095a5a6ed7cba2f17a266bf4 \
--hash=sha256:c2a2a42198b696a6f48fad91709afb55176e66a5e566131219dba372fb7f8c59 \
--hash=sha256:caeb583deeb5168e694b65cda8b4ee62abedfa66cf88488734466f2366b9c4e0 \
--hash=sha256:cb014d58140a38135f16064c74c652ed57aa0b75cbf8bb59cac821f7edb5334e \
--hash=sha256:ccf41f2efdf56994d22d73bef4ced1052161958169428d06ba9724ea9e9a64be \
--hash=sha256:cd7e9857e5e63738b9d9fd707bc1f59c8b09e5177726d23664db393c59bb08bd \
--hash=sha256:d76ac49f929aecaf82d83250b8347e099d7aecba0f4726c1d9b6df3b8bb5fe18 \
--hash=sha256:d7e5c9973aa04c95650c96e5f5ad865fbf42d62079163ecfab1e01cbc2504c22 \
--hash=sha256:dcf076a4474fe0d7367e5bbf5b052c7284fa1feca729c04176ce513521afd8a0 \
--hash=sha256:e3297a6a4059b4acc3a1e9a8b04741f240a80044eef08ebd32e8b5bcdddce75b \
--hash=sha256:ee08ebfa58f6e1aeff5697ab9582105bb620008c1caafb681e4c557e7483027b \
--hash=sha256:ef3048ef05dbb552b89817713d9cac912e00d0fde4a3105c00d29e52e10c89af \
--hash=sha256:fd1e3094f42d806d3d7c79162fc59e5910fcbe3a7360c385b8da969bc4493745
# via weasyprint
identify==2.6.19 \
--hash=sha256:20e6a87f786f768c092a721ad107fc9df0eb89347be9396cadf3f4abbd1fb78a \
--hash=sha256:6be5020c38fcb07da56c53733538a3081ea5aa70d36a156f83044bfbf9173842
# via pre-commit
nodeenv==1.10.0 \
--hash=sha256:5bb13e3eed2923615535339b3c620e76779af4cb4c6a90deccc9e36b274d3827 \
--hash=sha256:996c191ad80897d076bdfba80a41994c2b47c68e224c542b48feba42ba00f8bb
# via pre-commit
numpy==2.4.6 \
--hash=sha256:001fbb8e08d942dd57599e781f2472269ee7f2755fae407b4f67b2f0b17da3f1 \
--hash=sha256:0280e0356c0829a18d9de1cb7eee50ec22ca639878d7240307ca0943d73cd2c4 \
--hash=sha256:043191bfa8eab18c776647b62723ac9dddece59743b13f49b2016094129c2b3f \
--hash=sha256:06ca2f61ec4385a07a6977c55ba998a4466c123642b4a32694d3128fce18c079 \
--hash=sha256:0a041d3d761dc3c35cc56ce0351506a02bcbc25f7b169f652435141a17db9096 \
--hash=sha256:0ab0a9c4ffb1a6d95ef519fe4247dba8eb6b18ad93999f76b7f657039acabd47 \
--hash=sha256:0c9136e14ed34a9e343a31c533d78a9813a69a3148332bce5e9821cb2f996e66 \
--hash=sha256:110f8b71aacb688ec69062bb7f6938a0f8acb01b7c1c4beb453c65b6d234584d \
--hash=sha256:112b06a867b235ef466ed3508ddf0238050df9c727cafb5301ac385b899189a1 \
--hash=sha256:17f9ade344e7d9b464a084d69bcf18fc691cb1db67c62ed80820bf4926d78f0e \
--hash=sha256:1e254a00cdf42b1e4d5b3d68d33af63268d41340d8885df2ab6470f2e1500147 \
--hash=sha256:1e978ec1e8bd0e0e4de6bb75de9d30cbb74db6b6a2bb727618613703ca0167dd \
--hash=sha256:25c692919ac5a01f170a3bfcd62d745b24fd095c353d50812637d6fcab442e75 \
--hash=sha256:260a5d70215b61ab4fadf5c7baacd64821842975eea312125ed3c39a6391b063 \
--hash=sha256:2803abfebfc990042cd494d8ce2d5f82e9d847af6d35ec486923aa19dbad5e73 \
--hash=sha256:29a287e0cf63ff528da061de6b9f64a4618da591ca1046aafc54062e40ca7eab \
--hash=sha256:29cb7f67d10b479ff07c17d33e39f78c07f71c40ef30d63c153d340e96cd3fb4 \
--hash=sha256:3213d622a0283a39a93d188f3cf72b26862df52fbb4ca3697f51705016523d41 \
--hash=sha256:33111801a01c12a8a1e3721f0a9232f8cfc8ae2c6b7098167e6f623c6073f402 \
--hash=sha256:357cc07a6d7b0b182ff02249616a03742827ebb1277546b5c7cd7f7620a45698 \
--hash=sha256:38efbc8de75c7a0fc1ac190162d892787f3f47b57cc291231aafee36b80982b7 \
--hash=sha256:4081eb135ac24158bd51cdfbef16f1c64df7063b1143f24731387137c092bec8 \
--hash=sha256:40fdc1ae7125e518ea98e53e69a4ebc27e1fd50510c47b7ea130cf21e5e1d42b \
--hash=sha256:4cfe66903cc32a9921a6733d96b19bb6abf310397581bbad89c228f5abaf0ee8 \
--hash=sha256:511dbaf848decaaaf4b4ca48032619fb3138710c4bf7da7617765edad1ef96b0 \
--hash=sha256:55cced7c52e981362f708ad635198e97a752dfba412cc03c23bbf3bd8d5cd662 \
--hash=sha256:56b39e5e0622a09a25bf5baf62f4bcf0cb8a41ae6e2819cf49bbc5a74c083f91 \
--hash=sha256:5dbbdb29840ca3d91ee0fece42fc29278886d908280bfec0a5846c6f901a3eb0 \
--hash=sha256:5f9fb9157b4ce2971008323afe46053787b526ef624fea915b261468a8421a0f \
--hash=sha256:6180d8b35af935aed8ece3a85e0a43f87393ae0ac87c8d2c8bd2c993f7270ef3 \
--hash=sha256:68a5124b13fa6cc2086764a20005d30bc0548146f7f5322f02fce212ca14317f \
--hash=sha256:68bb27509ac1b9a3443094260f6326150663b06abe40b73a2f81160623da5b67 \
--hash=sha256:6f41ae150c4e32db4f3310cdaf64b1593a03dbabe29eec77fc9b50fe64061df6 \
--hash=sha256:7265a2f3d436e54ef9f2b52b5c937e6be778781bd97a590319d7348f1c1ca997 \
--hash=sha256:72fbe16c6fac95aedf5937fa873445cec2110be35d8a4e9433d7501fd98dae6b \
--hash=sha256:7d92c3819208a60205a12a245c91ad70cb0a85336659b19b834205573ac8456e \
--hash=sha256:8155154c7c691289fe18f510b5d4657c68c67989f293f0535a91360392ff6538 \
--hash=sha256:81a1cca95ed5bb92aa8b10dd2cdc9a0d3853a50fad926c28b5d7e8ea54389627 \
--hash=sha256:89cd468399cfd2504718f0ba50e410dca55a170b61a02ad92bb18c8a65186e93 \
--hash=sha256:8ad03c0965fb3c692200e74d458ca28c1dbb4ce96f9a479a8aa041ad5fabca02 \
--hash=sha256:90f9849678c75fe7afa2d348ac842c168b0a4d3d61919687216dfc547976d853 \
--hash=sha256:948424b06129ce883307e8cff868c31396d8dc7630a59c61d70d98dbe70f222c \
--hash=sha256:9cd5ffd25db4e7ba6a375693b3fc0fc1791ec636c17db3720da19bde7180ec43 \
--hash=sha256:a0df0043bdb289bde1f62da130d20df23d58b45429f752bc7a8fc5325a225ecd \
--hash=sha256:a2c306dea656c12c68f51f4cea133cbe78ca7435eb28c735eac1d3ebe73be6e8 \
--hash=sha256:a7830bab239b79cda9c08c2da014761cafb48da6150e1da17ac06283f43b6089 \
--hash=sha256:a7c711e21628b52034bb5ab8d1bce291f752fcc5e92accc615778acee1ff4778 \
--hash=sha256:aaf159caa35993cb1f56fb9b8e4610d35758e7ca005412eb1daa856a78c9c4b1 \
--hash=sha256:ae506e6902902557576a26ff33eda8695e7ecb3cb36c3b573a0765dee114ebdb \
--hash=sha256:b507f5c4c1d508876d1819b6bf9a49d365b96320b5d4993426b33a23ca4b8261 \
--hash=sha256:bf162abab1c1a736333192707cef898e735a5ca00f38f27eeedf44b39d9e85eb \
--hash=sha256:c1a2af6c6ef86344a6b0db6b97834208bf598db514f2b155042439b62605601a \
--hash=sha256:c2d37ab77531417474168eb79d6d80b14f821a966818505d03013d0833edb7a8 \
--hash=sha256:c4fc99836233ea196540b17ab0983aff60ed07941751930f5f4d05bc3b3b7359 \
--hash=sha256:d581b735e177fdcdce6fed8e7e8880a3fb6ee4e3653a3ac6af01c6f4c03effc5 \
--hash=sha256:d6da64deb6b8ed903e7560180a92f2d804ee1ba5eeb849ac2748b8c1aba1f6d7 \
--hash=sha256:d8e8286dd7cea7895157318d1b91cdacac64c479f3cbc8dce548331728484751 \
--hash=sha256:ddea102b48f9e339f3948bf22040944184627a30fdf7f858667673b9c5f033c8 \
--hash=sha256:dfa20cc6ca228e6b155b11da03825975ce66aea520985dbbddf0f2a5a495c605 \
--hash=sha256:e3e5193ef5a3dc73bceee50f7fdc2c90dbb76c42df8d8fae3d1067a583df579e \
--hash=sha256:e3eeb0aabd6bd5ce64faae67e9935203a6991b4bc2a485a767fbafb2c5125f45 \
--hash=sha256:e5805d5a22fd19c8ccff10a9561f9df94436b0545619ea579db2d3c35294bce2 \
--hash=sha256:e85b752a1e912b70eaad4fafbd4d1238007ab221de2009b9a2f5ae7461239895 \
--hash=sha256:eaf7fa2de5c0be8ae6ff8e9bea2ccd725e980541244521d8d4b5f3354a27babe \
--hash=sha256:ebfb099f8dcf083deef3ac1ca4c1503f387cf76296fcb3816b66f5ecb5f54fdb \
--hash=sha256:ece3d2cfe132e7d51f44a832b303895e6f2d499c5e74dfbdb06ee246147a304a \
--hash=sha256:ed9749eef4cbd126da3dc1d6bcb3a57f5eb7ac6a6484146bdbf743f552dfc577 \
--hash=sha256:ede83e07a75dd06bc501566c1eca2afc0d61677c1472ac9ad93fdee6e638a48d \
--hash=sha256:ef4aea96ce4d3b074422cb4f2f64e216bf9e213004bb58ecfdf50ea02ea8eb9a \
--hash=sha256:f3a3570c4a2a16746ac2c31a7c7c7b0c186b95ce902e33db6f28094ed7387dda \
--hash=sha256:f407cb6b8e9d6d8c626bc73c945db1706035af8fd632295547bf1c9e46d092d6 \
--hash=sha256:f74a575920ab21fe304421a3fc28793d82e299cae9eccb37084e9fc7f3617c20
# via opencv-python-headless
opencv-python-headless==5.0.0.93 \
--hash=sha256:030ca5e0837a2963ab36ef896baa9767eb8d2b83353fb28af5a521e40dd8756f \
--hash=sha256:09a872a157c1376ab922a69bbf22f9a95bcc7b658a9d8b436a60212b02b2eeb4 \
--hash=sha256:10818d91510e05c04568ae12b5cd120779c70c01bf897b001a6221fe430df80f \
--hash=sha256:1e55af3abfb462eeeabe5c775f12bdb36216d8a93a3583d69e6bd6e1d6ba7d00 \
--hash=sha256:829717b6a95554f273e49e357cee3b3a2a26b6f4842fbc1bed2b45bdd8f87e0e \
--hash=sha256:840bd717c21e5c11cadadc022a823315ea417f961213d06b4df010e019eb16f4 \
--hash=sha256:b82f9831daab90b725c7c1ee1b36cb5732c367096ac76d119e64e14eb70d5f3c \
--hash=sha256:c6bcd96b185975ea240d22cfdb15a1f6d080cc95264cfbe2621f21bb144d89b9 \
--hash=sha256:ed709fdf9aa0bd1f2ed8549e71d19449b03a675bb581eb292285f6861953be37
# via -r .github/scripts/requirements_dev.in
pdf2image==1.17.0 \
--hash=sha256:eaa959bc116b420dd7ec415fcae49b98100dda3dd18cd2fdfa86d09f112f6d57 \
--hash=sha256:ecdd58d7afb810dffe21ef2b1bbc057ef434dabbac6c33778a38a3f7744a27e2
# via -r .github/scripts/requirements_dev.in
pillow==12.3.0 \
--hash=sha256:00808c5e14ef63ac5161091d242999076604ff74b883423a11e5d7bbb38bf756 \
--hash=sha256:04f01d28a6aaff387bf842a13be313df23ba0597a44f1a976c9feb3c6ff4711a \
--hash=sha256:06ff022112bc9cbf83b60f8e028d94ad87b60621706487e65f673de61610ab59 \
--hash=sha256:0740a512dc522224c77d9aa5a8d70d8b7d73fb91f2c21125d8d025d3b8990e45 \
--hash=sha256:0847a763afefb695bc912d7c131e7e0632d4edc1d8698f58ddabec8e46b8b6d3 \
--hash=sha256:0dd2064cbc55aaec028ef5fbb60fa47bb6c3e7918e07ff17935284b227a9d2df \
--hash=sha256:0feb2e9d6ad6c9e3c06effe9d00f3f1e618a6643273576b016f591e9315a7139 \
--hash=sha256:10e41f0fbf1eec8cfd234b8fe17a4caac7c9d0db4c204d3c173a8f9f6ef3232b \
--hash=sha256:1182d52bc2d5e5d7d0949503aa7e36d12f42205dc287e4883f407b1988820d39 \
--hash=sha256:164b31cd1a0490ab6efae01aa5df49da7061be0af1b30e035b6e9a1bfe34ee6e \
--hash=sha256:1657923d2d45afb66526e5b933e5b3052e6bdea196c90d3abb2424e18c77dae8 \
--hash=sha256:186941b6aef820ad110fb01fb06eb925374dc3a21b17e37ec9a53b250c6fe2d1 \
--hash=sha256:1cca606cd25738df4ed873d5ad46bbdb3d83b5cbca291f6b4ff13a4df6b0bbe8 \
--hash=sha256:21900ce7ba264168cd50defae43cd75d25c833ad4ad6e73ffc5596d12e25ac89 \
--hash=sha256:236ff70b9312fb68943c703aa842ca6a758abfa45ac187a5e7c1452e96ef72b5 \
--hash=sha256:23aceaa007d6172b02c277f0cd359c79492bbb14f7072b4ede9fbcaf20648130 \
--hash=sha256:23d27a3e0307ec2244cc51e7287b919aa68d097504ebe19df4e76a98a3eea5bd \
--hash=sha256:24870b09b224f7ae3c39ed07d10e819d06f8720bc551847b1d623832b5b0e28d \
--hash=sha256:251bf95b67017e27b13d82f5b326234ca62d70f9cf4c2b9032de2358a3b12c7b \
--hash=sha256:25b9b82bb22e6e2b3cd07b39c68b7b862001226cb3dff7130d1cb914121b39ed \
--hash=sha256:28ce87c5ab450a9dd970b52e5aca5fe63ed432d18a2eaddd1979a00a1ba24ace \
--hash=sha256:300557495eb45ebb8aec96c2da9c4be642fbf7cd937278b4013ba894ea8eb0eb \
--hash=sha256:30f2aa603c41533cc25c05acd0da21636e84a315768feb631c937177db558931 \
--hash=sha256:331b624368d4f1d069149002f25f44bc61c8919ce8ddb3c45bdad8f6e2d89510 \
--hash=sha256:37d6d0a00072fd2948eb22bce7e1475f34569d90c87c59f7a2ec59541b77f7a6 \
--hash=sha256:37dc8f7bbb66efe481bb60defacef820c950c24713fb44962ed6aa2a50966de1 \
--hash=sha256:3b8182a766685eaa002637e28b4ec8d6b18819a0c71f579bf0dbaa5830297cce \
--hash=sha256:3edce1d53195db527e0191f84b71d02022de0540bf43a16ed734ed7537b07385 \
--hash=sha256:446c34dcc4324b084a53b705127dc15717b22c5e140ae0a3c38349d4efec071e \
--hash=sha256:4998562bf62a445225f22e07c896bb04b35b1b1f2eb6d760584c9c51d7a5f78c \
--hash=sha256:4b0a7fe987b14c31ebda6083f74f22b561fd3739bc0ac51e019622e3d72668c7 \
--hash=sha256:4e8c2a84d977f50b9daed6eeaf3baef67d00d5d74d932288f02cb94518ee3ace \
--hash=sha256:4f883547d4b7f0495ebe7056b0cc2aea76094e7a4abc8e933540f3271df27d9c \
--hash=sha256:514435a37670e3e5e08f3945b68718b6ed329bb84367777e16f9f4dfe1e61a0f \
--hash=sha256:53aa02d20d10c3d814d536aa4e5ac9b84ca0ff5a88377963b085ad6822f93e64 \
--hash=sha256:5594fc43d548a7ed94949d139aa1341b270f1863f11cfd37f5a6c8b778a6b67f \
--hash=sha256:571b9fcb07b97ef3a492028fb3d2dc0993ca23a06138b0315286566d29ef718a \
--hash=sha256:57b3d78c95ba9059768b10e28b813002261d3f3dfc55cc48b0c988f625175827 \
--hash=sha256:5afb51d599ea772b8365ae807ae557f18bccfe46ab261fd1c2a9ed700fc6eb17 \
--hash=sha256:6b02afb9b97f65fbca5f31db6a2a3ba21aa93030225f150fa3f249717e938fb4 \
--hash=sha256:6c0016e7b354317c4e9e525b937ac8596c38d2d232b419529b9cd7a1cd46e39a \
--hash=sha256:71d6097b330eea8fd15097780c8e89cb1a8ce7838669f48c5bacd6f663dd4701 \
--hash=sha256:756c768d0c9c2955feb7a56c37ea24aea2e369f8d36a88da270b6a9f19e62b5e \
--hash=sha256:78cb2c6865a35ab8ff8b75fd122f6033b92a62c82801110e48ddd6c936a45d91 \
--hash=sha256:7a743ff716f746fc19a9557f60dab1600d4613255f8a7aeb3cdde4db7eb15a66 \
--hash=sha256:85f998ea1848bc6757289e739cfbdda3a04adfd58b02fc018ce54d754a5ce468 \
--hash=sha256:8728f216dcdb6e6d555cf971cb34076139ad74b31fc2c14da4fafc741c5f6217 \
--hash=sha256:877c3f311ff35410f690861c4409e7ccbf0cd2f878e50628a28e5a0bb689e658 \
--hash=sha256:8cd2f7bdda092d99c9fc2fb7391354f306d01443d22785d0cbfafa2e2c8bb418 \
--hash=sha256:8e95e1385e4998ae9694eeaa4730ba5457ff61185b3a55e2e7bea0880aef452a \
--hash=sha256:962864dc93511324d51ddbb5b9f8731bf71675b93ca612a07441896f4688fb8c \
--hash=sha256:9cf95fe4d0f84c82d282745d9bb08ad9f926efa00be4697e767b814ce40d4330 \
--hash=sha256:9e881fca225083806662a5c43d627d215f258ff43c890f831966c7d7ba9c7402 \
--hash=sha256:a2b55dd6b2a4c4b7d87ffa56bdb33fdc5fdb9a462173861a7bc097f17d91cb09 \
--hash=sha256:a45650e8ce7fafffd731db8550230db6b0d306d181a90b67d3e6bca2f1990930 \
--hash=sha256:a876864214e136f0eb367788dbd7df045f4806801518e2cfe9e13229cfe06d8f \
--hash=sha256:ae26d61dfa7a47befdc7572b521024e8745f3d809bd95ca9505a7bba9ef849ec \
--hash=sha256:af8d94b0db561cf68b88a267c5c44b49e134f525d0dc2cb7ed413a66bc23559a \
--hash=sha256:b343699e8308bdc51978310e1c959c584e7869cc8c40780058c87da7781a1e94 \
--hash=sha256:b3c777e849237620b022f7f297dd67705f9f5cf1685f09f02e46f93e92725468 \
--hash=sha256:b629de27fda84b42cde7edef0d85f13b958b47f6e9bbcbba9b673c562a89bd8b \
--hash=sha256:ba09209fbe443b4acccebe845d8a138b89a8f4fbaeedd44953490b5315d5e965 \
--hash=sha256:ba54cfebe86920a559a7c4d6b9050791c20513650a1952ebe3368c7dc70306f8 \
--hash=sha256:bcb46e2f9feff8d06323983bd83ed00c201fdcab3d74973e7072a889b3979fcd \
--hash=sha256:bcc33feacfaefce60c12fd500a277533bdc02b10a19f7f6d348763d8140bbba7 \
--hash=sha256:bf16ba1b4d0b6b7c8e534936632270cf70eb00dbe09005bc345b2677b726855c \
--hash=sha256:cf1845d02ad822a369a49f2bb9345b1614744267682e7a03527dc3bf6eea1777 \
--hash=sha256:d69141514cc30b774ceea5e3ed3a6635c8d8a96edf664689b890f4089111fb35 \
--hash=sha256:d9c7f76c0673154f044e9d78c8655fb4213f6ca31a836df48b40fe5d187717b9 \
--hash=sha256:dbce0b29841537a2fa4a214c2bbf14de3587c9680caa9b4e217568472490b28f \
--hash=sha256:dc624f6bc473dacdf7ef7eb8678d0d08edf15cd94fad6ae5c7d6cc67a4e4902f \
--hash=sha256:e158cb00350dc278f3b91551101aa7d12415a66ebf2c91d8d5ac14e56ddd3ad0 \
--hash=sha256:e491916b378fba47242221bb9ead245211b70d504f495d105d17b14a24b4907c \
--hash=sha256:e795b7eb908249c4e43c7c99fac7c2c75dab0c43566e37db472a355f63693d71 \
--hash=sha256:e7e480451b9fa137494bccd3a7d69adbe8ac65a87d97be61e11f1b1050a5bac3 \
--hash=sha256:e91206ee562682b51b98ef4b26a6ef48fd84e15fd4c4bc5ec768eb641d206838 \
--hash=sha256:e9871b1ffbfa9656b60aeee92ed5136a5742696006fa322b29ea3d8da0ecc9cf \
--hash=sha256:e9aeb04d6aef139de265b29683e119b638208f88cf73cdd1658aa07221165321 \
--hash=sha256:ebaea975e03d3141d9d3a507df75c9b3ec90fa9d2ffd07567b3a978d9d790b26 \
--hash=sha256:f0606c8bf2cdefea14a43530f7657cbbb7ecf1c4222512492ef4a4434a9501ec \
--hash=sha256:f13c32a3abd6079a66d9526e18dad9b6d280384d49d7c54040cd57b6424041d9 \
--hash=sha256:f7401aebd7f581d7f83a439d87d474999317ee099218e5ad25d125290990ba65 \
--hash=sha256:fa4ecea169a355be7a3ade2c783e2ed12f0e40d2c5621cda8b3297faf7fbb9f5 \
--hash=sha256:fbd139c8447d25dd750ab79ee274cc5e1fe80fc56340ab10b18a195e1b6eca3e \
--hash=sha256:fdafc9cce40277e0f7a0feabce0ee50dd2fa1800f3b38015e51296b5e814048d \
--hash=sha256:fe3cca2e4e8a592be0f269a1ca4835c25199d9f3ce815c8491048f785b0a0198 \
--hash=sha256:ffd0c5368496f41b0944be820fcb7a838aa6e623d250b01acf2643939c3f99d7
# via
# -r .github/scripts/requirements_dev.in
# pdf2image
# weasyprint
platformdirs==4.10.0 \
--hash=sha256:31e761a6a0ca04faf7353ea759bdba55652be214725111e5aac52dfa29d4bef7 \
--hash=sha256:fb516cdb12eb0d857d0cd85a7c57cea4d060bee4578d6cf5a14dfdf8cbf8784a
# via
# python-discovery
# virtualenv
pre-commit==4.6.0 \
--hash=sha256:718d2208cef53fdc38206e40524a6d4d9576d103eb16f0fec11c875e7716e9d9 \
--hash=sha256:e2cf246f7299edcabcf15f9b0571fdce06058527f0a06535068a86d38089f29b
# via -r .github/scripts/requirements_dev.in
pycparser==3.0 \
--hash=sha256:600f49d217304a5902ac3c37e1281c9fe94e4d0489de643a9504c5cdfdfc6b29 \
--hash=sha256:b727414169a36b7d524c1c3e31839a521725078d7b2ff038656844266160a992
# via cffi
pydyf==0.12.1 \
--hash=sha256:ea25b4e1fe7911195cb57067560daaa266639184e8335365cc3ee5214e7eaadc \
--hash=sha256:fbd7e759541ac725c29c506612003de393249b94310ea78ae44cb1d04b220095
# via weasyprint
pyphen==0.17.2 \
--hash=sha256:3a07fb017cb2341e1d9ff31b8634efb1ae4dc4b130468c7c39dd3d32e7c3affd \
--hash=sha256:f60647a9c9b30ec6c59910097af82bc5dd2d36576b918e44148d8b07ef3b4aa3
# via weasyprint
python-discovery==1.4.4 \
--hash=sha256:5cad33982d412c1f3ffb8f9ca4ea292c9680bca3942451d30b69c37fce53a4a3 \
--hash=sha256:abebe9120b43453b68c908acfb1e72a19d1a959ed2cb620ad38fc57d08056dbe
# via virtualenv
pyyaml==6.0.3 \
--hash=sha256:00c4bdeba853cc34e7dd471f16b4114f4162dc03e6b7afcc2128711f0eca823c \
--hash=sha256:0150219816b6a1fa26fb4699fb7daa9caf09eb1999f3b70fb6e786805e80375a \
--hash=sha256:02893d100e99e03eda1c8fd5c441d8c60103fd175728e23e431db1b589cf5ab3 \
--hash=sha256:02ea2dfa234451bbb8772601d7b8e426c2bfa197136796224e50e35a78777956 \
--hash=sha256:0f29edc409a6392443abf94b9cf89ce99889a1dd5376d94316ae5145dfedd5d6 \
--hash=sha256:10892704fc220243f5305762e276552a0395f7beb4dbf9b14ec8fd43b57f126c \
--hash=sha256:16249ee61e95f858e83976573de0f5b2893b3677ba71c9dd36b9cf8be9ac6d65 \
--hash=sha256:1d37d57ad971609cf3c53ba6a7e365e40660e3be0e5175fa9f2365a379d6095a \
--hash=sha256:1ebe39cb5fc479422b83de611d14e2c0d3bb2a18bbcb01f229ab3cfbd8fee7a0 \
--hash=sha256:214ed4befebe12df36bcc8bc2b64b396ca31be9304b8f59e25c11cf94a4c033b \
--hash=sha256:2283a07e2c21a2aa78d9c4442724ec1eb15f5e42a723b99cb3d822d48f5f7ad1 \
--hash=sha256:22ba7cfcad58ef3ecddc7ed1db3409af68d023b7f940da23c6c2a1890976eda6 \
--hash=sha256:27c0abcb4a5dac13684a37f76e701e054692a9b2d3064b70f5e4eb54810553d7 \
--hash=sha256:28c8d926f98f432f88adc23edf2e6d4921ac26fb084b028c733d01868d19007e \
--hash=sha256:2e71d11abed7344e42a8849600193d15b6def118602c4c176f748e4583246007 \
--hash=sha256:34d5fcd24b8445fadc33f9cf348c1047101756fd760b4dacb5c3e99755703310 \
--hash=sha256:37503bfbfc9d2c40b344d06b2199cf0e96e97957ab1c1b546fd4f87e53e5d3e4 \
--hash=sha256:3c5677e12444c15717b902a5798264fa7909e41153cdf9ef7ad571b704a63dd9 \
--hash=sha256:3ff07ec89bae51176c0549bc4c63aa6202991da2d9a6129d7aef7f1407d3f295 \
--hash=sha256:41715c910c881bc081f1e8872880d3c650acf13dfa8214bad49ed4cede7c34ea \
--hash=sha256:418cf3f2111bc80e0933b2cd8cd04f286338bb88bdc7bc8e6dd775ebde60b5e0 \
--hash=sha256:44edc647873928551a01e7a563d7452ccdebee747728c1080d881d68af7b997e \
--hash=sha256:4a2e8cebe2ff6ab7d1050ecd59c25d4c8bd7e6f400f5f82b96557ac0abafd0ac \
--hash=sha256:4ad1906908f2f5ae4e5a8ddfce73c320c2a1429ec52eafd27138b7f1cbe341c9 \
--hash=sha256:501a031947e3a9025ed4405a168e6ef5ae3126c59f90ce0cd6f2bfc477be31b7 \
--hash=sha256:5190d403f121660ce8d1d2c1bb2ef1bd05b5f68533fc5c2ea899bd15f4399b35 \
--hash=sha256:5498cd1645aa724a7c71c8f378eb29ebe23da2fc0d7a08071d89469bf1d2defb \
--hash=sha256:5cf4e27da7e3fbed4d6c3d8e797387aaad68102272f8f9752883bc32d61cb87b \
--hash=sha256:5e0b74767e5f8c593e8c9b5912019159ed0533c70051e9cce3e8b6aa699fcd69 \
--hash=sha256:5ed875a24292240029e4483f9d4a4b8a1ae08843b9c54f43fcc11e404532a8a5 \
--hash=sha256:5fcd34e47f6e0b794d17de1b4ff496c00986e1c83f7ab2fb8fcfe9616ff7477b \
--hash=sha256:5fdec68f91a0c6739b380c83b951e2c72ac0197ace422360e6d5a959d8d97b2c \
--hash=sha256:6344df0d5755a2c9a276d4473ae6b90647e216ab4757f8426893b5dd2ac3f369 \
--hash=sha256:64386e5e707d03a7e172c0701abfb7e10f0fb753ee1d773128192742712a98fd \
--hash=sha256:652cb6edd41e718550aad172851962662ff2681490a8a711af6a4d288dd96824 \
--hash=sha256:66291b10affd76d76f54fad28e22e51719ef9ba22b29e1d7d03d6777a9174198 \
--hash=sha256:66e1674c3ef6f541c35191caae2d429b967b99e02040f5ba928632d9a7f0f065 \
--hash=sha256:6adc77889b628398debc7b65c073bcb99c4a0237b248cacaf3fe8a557563ef6c \
--hash=sha256:79005a0d97d5ddabfeeea4cf676af11e647e41d81c9a7722a193022accdb6b7c \
--hash=sha256:7c6610def4f163542a622a73fb39f534f8c101d690126992300bf3207eab9764 \
--hash=sha256:7f047e29dcae44602496db43be01ad42fc6f1cc0d8cd6c83d342306c32270196 \
--hash=sha256:8098f252adfa6c80ab48096053f512f2321f0b998f98150cea9bd23d83e1467b \
--hash=sha256:850774a7879607d3a6f50d36d04f00ee69e7fc816450e5f7e58d7f17f1ae5c00 \
--hash=sha256:8d1fab6bb153a416f9aeb4b8763bc0f22a5586065f86f7664fc23339fc1c1fac \
--hash=sha256:8da9669d359f02c0b91ccc01cac4a67f16afec0dac22c2ad09f46bee0697eba8 \
--hash=sha256:8dc52c23056b9ddd46818a57b78404882310fb473d63f17b07d5c40421e47f8e \
--hash=sha256:9149cad251584d5fb4981be1ecde53a1ca46c891a79788c0df828d2f166bda28 \
--hash=sha256:93dda82c9c22deb0a405ea4dc5f2d0cda384168e466364dec6255b293923b2f3 \
--hash=sha256:96b533f0e99f6579b3d4d4995707cf36df9100d67e0c8303a0c55b27b5f99bc5 \
--hash=sha256:9c57bb8c96f6d1808c030b1687b9b5fb476abaa47f0db9c0101f5e9f394e97f4 \
--hash=sha256:9c7708761fccb9397fe64bbc0395abcae8c4bf7b0eac081e12b809bf47700d0b \
--hash=sha256:9f3bfb4965eb874431221a3ff3fdcddc7e74e3b07799e0e84ca4a0f867d449bf \
--hash=sha256:a33284e20b78bd4a18c8c2282d549d10bc8408a2a7ff57653c0cf0b9be0afce5 \
--hash=sha256:a80cb027f6b349846a3bf6d73b5e95e782175e52f22108cfa17876aaeff93702 \
--hash=sha256:b30236e45cf30d2b8e7b3e85881719e98507abed1011bf463a8fa23e9c3e98a8 \
--hash=sha256:b3bc83488de33889877a0f2543ade9f70c67d66d9ebb4ac959502e12de895788 \
--hash=sha256:b865addae83924361678b652338317d1bd7e79b1f4596f96b96c77a5a34b34da \
--hash=sha256:b8bb0864c5a28024fac8a632c443c87c5aa6f215c0b126c449ae1a150412f31d \
--hash=sha256:ba1cc08a7ccde2d2ec775841541641e4548226580ab850948cbfda66a1befcdc \
--hash=sha256:bdb2c67c6c1390b63c6ff89f210c8fd09d9a1217a465701eac7316313c915e4c \
--hash=sha256:c1ff362665ae507275af2853520967820d9124984e0f7466736aea23d8611fba \
--hash=sha256:c2514fceb77bc5e7a2f7adfaa1feb2fb311607c9cb518dbc378688ec73d8292f \
--hash=sha256:c3355370a2c156cffb25e876646f149d5d68f5e0a3ce86a5084dd0b64a994917 \
--hash=sha256:c458b6d084f9b935061bc36216e8a69a7e293a2f1e68bf956dcd9e6cbcd143f5 \
--hash=sha256:d0eae10f8159e8fdad514efdc92d74fd8d682c933a6dd088030f3834bc8e6b26 \
--hash=sha256:d76623373421df22fb4cf8817020cbb7ef15c725b9d5e45f17e189bfc384190f \
--hash=sha256:ebc55a14a21cb14062aa4162f906cd962b28e2e9ea38f9b4391244cd8de4ae0b \
--hash=sha256:eda16858a3cab07b80edaf74336ece1f986ba330fdb8ee0d6c0d68fe82bc96be \
--hash=sha256:ee2922902c45ae8ccada2c5b501ab86c36525b883eff4255313a253a3160861c \
--hash=sha256:efd7b85f94a6f21e4932043973a7ba2613b059c4a000551892ac9f1d11f5baf3 \
--hash=sha256:f7057c9a337546edc7973c0d3ba84ddcdf0daa14533c2065749c9075001090e6 \
--hash=sha256:fa160448684b4e94d80416c0fa4aac48967a969efe22931448d853ada8baf926 \
--hash=sha256:fc09d0aa354569bc501d4e787133afc08552722d3ab34836a80547331bb5d4a0
# via pre-commit
tinycss2==1.5.1 \
--hash=sha256:3415ba0f5839c062696996998176c4a3751d18b7edaaeeb658c9ce21ec150661 \
--hash=sha256:d339d2b616ba90ccce58da8495a78f46e55d4d25f9fd71dfd526f07e7d53f957
# via
# cssselect2
# weasyprint
tinyhtml5==2.1.0 \
--hash=sha256:60a50ec3d938a37e491efa01af895853060943dcebb5627de5b10d188b338a67 \
--hash=sha256:6e11cfff38515834268daf89d5f85bbde0b6dd02e8d9e212d1385c2289b89f0a
# via weasyprint
unoserver==3.7 \
--hash=sha256:b05f9578506ac7374ae1b314c3a79528636c542ac78220a9ce99110584ca424b \
--hash=sha256:fc44e6808071c9d2957e705ecf1742cea8a582aa5d5cc23babf36bb332ec6e8e
# via -r .github/scripts/requirements_dev.in
virtualenv==21.6.1 \
--hash=sha256:15f978b7cd329f24855ff4a0c4b4899cc7678589f49adbdcbbb4d3232e641128 \
--hash=sha256:afe991df855715a2b2f60edfcc0107ef95a79fdfd8cb4cdaa71603d1c12e463b
# via pre-commit
weasyprint==69.0 \
--hash=sha256:475951cfd917014de6d4d005caff48c6aa867e7e42b80cd5b16a0484a1609ee6 \
--hash=sha256:a7a32f39ca16bd82ef11de99c92ea4b5f14951c9033af035e451ce4f4ee0a88c
# via -r .github/scripts/requirements_dev.in
webencodings==0.5.1 \
--hash=sha256:a0af1213f3c2226497a97e2b3aa01a7e4bee4f403f95be16fc9acd2947514a78 \
--hash=sha256:b36a1c245f2d304965eb4e0a82848379241dc04b865afcc4aab16748587e1923
# via
# cssselect2
# tinycss2
# tinyhtml5
zopfli==0.4.3 \
--hash=sha256:0087c9a6f0c8a052be0f6d1a9bb71b6caffdd3e10201d6d6166e28d482cebe6d \
--hash=sha256:47604eee5c6704bdf0e94d8391fe3b74ddb2abd84128fbcfdc3ee0fc265feaef \
--hash=sha256:62248dbf8dbcbd588ee194b210e5be9fa80bce29641f55599d6d394bd2a9d8a3 \
--hash=sha256:628c3e941752880b3491db8d44163d0aedb221944e22a17187ff7fc549b050f6 \
--hash=sha256:769875152d0625c46707bcca57d4b2233fe653482067acd55fbf6ec525cb9bdc \
--hash=sha256:7e9703ca6e7ef66c8d05e0826b6f558b680c9db8206f84f05a3ee93430a12e42 \
--hash=sha256:7fa3c35193475290e3f007bbcdebdbae64ba2f012d75c632da0d727e1da50d5e \
--hash=sha256:88f4fbe429aad72bc206275d81fab11a097e0f951a5848d1f51083c37ea73073 \
--hash=sha256:921c2c9907f4364963848da5ad194b46d68865e07fdb975d04fd09bc42d47357 \
--hash=sha256:d3a50f91a13cea9bafe025de8fd87a005eb26de02a4f0c193127ddbf23ac8ebe \
--hash=sha256:d4f51dd1ab5312e837e2091284e0d9f1a138188f2e65812f9a5799dc02c45f94 \
--hash=sha256:eb0c9c1d40a8cb1d58762d7e57290ccb753e0828c4d01be8acb59aae5d0ca206 \
--hash=sha256:f2e0adcf7d36c6fd0dd36cc771ef7f0c5803a05666feafcd90d7170174a4148e
# via fonttools
# The following packages are considered to be unsafe in a requirements file:
pip==26.1.2 \
--hash=sha256:382ff9f685ee3bc25864f820aa50505825f10f5458ffff07e30a6d96e5715cab \
--hash=sha256:f49cd134c61cf2fd75e0ce2676db03e4054504a5a4986d00f8299ae632dc4605
# via -r .github/scripts/requirements_dev.in
setuptools==83.0.0 \
--hash=sha256:025bccbbf0fa05b6192bc64ae1e7b16e001fd6d6d4d5de03c97b1c1ade523bef \
--hash=sha256:29b23c360f22f414dc7336bb39178cc7bcbf6021ed2733cde173f09dba19abb3
# via -r .github/scripts/requirements_dev.in
.github/scripts/requirements_sync_readme.in
-2
View File
@@ -1,2 +0,0 @@
tomlkit
tomli-w
.github/scripts/requirements_sync_readme.txt
-14
View File
@@ -1,14 +0,0 @@
#
# This file is autogenerated by pip-compile with Python 3.13
# by the following command:
#
# pip-compile --generate-hashes --output-file='.github\scripts\requirements_sync_readme.txt' --strip-extras '.github\scripts\requirements_sync_readme.in'
#
tomli-w==1.2.0 \
--hash=sha256:188306098d013b691fcadc011abd66727d3c414c571bb01b1a174ba8c983cf90 \
--hash=sha256:2dd14fac5a47c27be9cd4c976af5a12d87fb1f0b4512f81d69cce3b35ae25021
# via -r .github/scripts/requirements_sync_readme.in
tomlkit==0.15.0 \
--hash=sha256:4dbc8f0fc024412b57ced8757ac7461305126a648ff8c2c807fcb8e133a78738 \
--hash=sha256:7d1a9ecba3086638211b13814ea79c90dd54dd11993564376f3aa92271f5c7a3
# via -r .github/scripts/requirements_sync_readme.in
.github/scripts/verify-updater-signatures.py
+6 -7
View File
@@ -4,19 +4,18 @@
Usage: verify-updater-signatures.py <dir-to-scan> [tauri.conf.json] Usage: verify-updater-signatures.py <dir-to-scan> [tauri.conf.json]
""" """
import binascii
import sys
import json
import base64 import base64
import binascii
import hashlib import hashlib
import json
import sys
from pathlib import Path from pathlib import Path
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PublicKey
from cryptography.exceptions import InvalidSignature from cryptography.exceptions import InvalidSignature
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PublicKey
ART_ROOT = Path(sys.argv[1]) ART_ROOT = Path(sys.argv[1])
CONF = Path( CONF = Path(sys.argv[2] if len(sys.argv) > 2 else "frontend/editor/src-tauri/tauri.conf.json")
sys.argv[2] if len(sys.argv) > 2 else "frontend/editor/src-tauri/tauri.conf.json"
)
def load_pubkey(): def load_pubkey():
.github/workflows/PR-Demo-Comment-with-react.yml
+1 -1
View File
@@ -264,7 +264,7 @@ jobs:
if: needs.check-comment.outputs.enable_prototypes == 'true' if: needs.check-comment.outputs.enable_prototypes == 'true'
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0 uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
with: with:
context: ./engine context: .
file: ./engine/Dockerfile file: ./engine/Dockerfile
push: true push: true
cache-from: type=gha,scope=stirling-pdf-engine cache-from: type=gha,scope=stirling-pdf-engine
.github/workflows/ai-engine.yml
+11
View File
@@ -31,6 +31,9 @@ jobs:
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0 uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with: with:
enable-cache: true enable-cache: true
cache-dependency-glob: |
engine/pyproject.toml
engine/uv.lock
cache-suffix: ai-engine cache-suffix: ai-engine
- name: Install Task - name: Install Task
@@ -96,6 +99,14 @@ jobs:
echo "============================================" echo "============================================"
exit 1 exit 1
- name: Build engine production image
if: always()
run: docker build --file engine/Dockerfile --tag stirling-pdf-engine:ci .
- name: Build engine development image
if: always()
run: docker build --file engine/Dockerfile.dev --tag stirling-pdf-engine-dev:ci .
- name: Remove engine check comment on success - name: Remove engine check comment on success
if: steps.engine-check.outcome == 'success' && github.event_name == 'pull_request' if: steps.engine-check.outcome == 'success' && github.event_name == 'pull_request'
continue-on-error: true continue-on-error: true
.github/workflows/backend-build.yml
+8 -6
View File
@@ -192,12 +192,14 @@ jobs:
retention-days: 3 retention-days: 3
if-no-files-found: warn if-no-files-found: warn
- name: Install defusedxml for coverage summary - name: Install uv
# coverage-summary.py parses JaCoCo XML through defusedxml to
# silence security scanners that pattern-match on the stdlib
# xml.etree.ElementTree.parse call.
if: always() && matrix.flavor == 'saas' if: always() && matrix.flavor == 'saas'
run: python -m pip install --quiet defusedxml uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with:
enable-cache: true
cache-dependency-glob: |
engine/pyproject.toml
engine/uv.lock
- name: JaCoCo coverage step summary - name: JaCoCo coverage step summary
# Only the saas leg posts the JUnit summary - it's a strict # Only the saas leg posts the JUnit summary - it's a strict
@@ -206,7 +208,7 @@ jobs:
# near-identical tables crowding out the aggregate report. # near-identical tables crowding out the aggregate report.
if: always() && matrix.flavor == 'saas' if: always() && matrix.flavor == 'saas'
run: | run: |
python scripts/coverage-summary.py \ uv run --project engine --locked --group tools python scripts/coverage-summary.py \
--title "Backend JUnit coverage (JDK ${{ matrix.jdk-version }})" \ --title "Backend JUnit coverage (JDK ${{ matrix.jdk-version }})" \
--jacoco "common=app/common/build/reports/jacoco/test/jacocoTestReport.xml" \ --jacoco "common=app/common/build/reports/jacoco/test/jacocoTestReport.xml" \
--jacoco "core=app/core/build/reports/jacoco/test/jacocoTestReport.xml" \ --jacoco "core=app/core/build/reports/jacoco/test/jacocoTestReport.xml" \
.github/workflows/build-enterprise.yml
+9 -8
View File
@@ -329,15 +329,16 @@ jobs:
egress-policy: audit egress-policy: audit
- name: Checkout repository - name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Set up Python - name: Install uv
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with: with:
python-version: "3.12" enable-cache: true
cache: "pip" cache-dependency-glob: |
cache-dependency-path: ./testing/cucumber/requirements.txt engine/pyproject.toml
engine/uv.lock
- name: Install behave test deps - name: Install behave test deps
run: | run: |
pip install --require-hashes --only-binary=:all: -r ./testing/cucumber/requirements.txt uv sync --project engine --locked --group cucumber
- name: Build the multi-node image - name: Build the multi-node image
working-directory: testing/compose working-directory: testing/compose
run: docker compose -f "$MN_COMPOSE" build run: docker compose -f "$MN_COMPOSE" build
@@ -360,10 +361,10 @@ jobs:
- name: Run multi-node regression (implemented guarantees) - name: Run multi-node regression (implemented guarantees)
working-directory: testing/cucumber working-directory: testing/cucumber
# -e overrides behave.ini's exclusion of features/multinode; ~@known_gap skips any tracked-gap scenarios. # -e overrides behave.ini's exclusion of features/multinode; ~@known_gap skips any tracked-gap scenarios.
run: python -m behave features/multinode -e "features/enterprise" --tags="~@known_gap ~@destructive" --no-capture -f plain run: uv run --project ../../engine --locked --group cucumber python -m behave features/multinode -e "features/enterprise" --tags="~@known_gap ~@destructive" --no-capture -f plain
- name: Run multi-node failover (destructive) - name: Run multi-node failover (destructive)
working-directory: testing/cucumber working-directory: testing/cucumber
run: python -m behave features/multinode -e "features/enterprise" --tags="@destructive ~@known_gap" --no-capture -f plain run: uv run --project ../../engine --locked --group cucumber python -m behave features/multinode -e "features/enterprise" --tags="@destructive ~@known_gap" --no-capture -f plain
- name: Dump node logs on failure - name: Dump node logs on failure
if: failure() if: failure()
working-directory: testing/compose working-directory: testing/compose
.github/workflows/check-generated-models.yml
+3
View File
@@ -34,6 +34,9 @@ jobs:
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0 uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with: with:
enable-cache: true enable-cache: true
cache-dependency-glob: |
engine/pyproject.toml
engine/uv.lock
cache-suffix: generated-models cache-suffix: generated-models
- name: Set up JDK 25 - name: Set up JDK 25
.github/workflows/check_toml.yml
+8 -5
View File
@@ -195,20 +195,23 @@ jobs:
console.log(`Reference file path: ${referenceFilePath}`); console.log(`Reference file path: ${referenceFilePath}`);
core.exportVariable("REFERENCE_FILE", referenceFilePath); core.exportVariable("REFERENCE_FILE", referenceFilePath);
- name: Set up Python - name: Install uv
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with: with:
python-version: "3.12" enable-cache: true
cache-dependency-glob: |
engine/pyproject.toml
engine/uv.lock
- name: Install Python dependencies - name: Install Python dependencies
run: | run: |
pip install --require-hashes --only-binary=:all: -r ./.github/scripts/requirements_sync_readme.txt uv sync --project engine --locked --group tools
- name: Run Python script to check files - name: Run Python script to check files
id: run-check id: run-check
run: | run: |
echo "Running Python script to check TOML files..." echo "Running Python script to check TOML files..."
python .github/scripts/check_language_toml.py \ uv run --project engine --locked --group tools python .github/scripts/check_language_toml.py \
--actor ${{ github.event.pull_request.user.login }} \ --actor ${{ github.event.pull_request.user.login }} \
--reference-file "${REFERENCE_FILE}" \ --reference-file "${REFERENCE_FILE}" \
--branch "pr-branch" \ --branch "pr-branch" \
.github/workflows/coverage-aggregate.yml
+9 -11
View File
@@ -57,15 +57,13 @@ jobs:
gradle-${{ runner.os }}-${{ runner.arch }}-jdk-25- gradle-${{ runner.os }}-${{ runner.arch }}-jdk-25-
gradle-${{ runner.os }}-${{ runner.arch }}- gradle-${{ runner.os }}-${{ runner.arch }}-
- name: Set up Python - name: Install uv
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with: with:
python-version: "3.12" enable-cache: true
cache-dependency-glob: |
- name: Install defusedxml for coverage scripts engine/pyproject.toml
# Both coverage-summary.py and coverage-matrix.py parse JaCoCo engine/uv.lock
# XML through defusedxml - see the script headers for context.
run: python -m pip install --quiet defusedxml
# Pattern matches every artifact this PR's producers might upload: # Pattern matches every artifact this PR's producers might upload:
# jacoco-exec-junit-jdk-25 (uploaded only by the saas # jacoco-exec-junit-jdk-25 (uploaded only by the saas
@@ -158,7 +156,7 @@ jobs:
# ("how much of the backend do real user flows cover?"). # ("how much of the backend do real user flows cover?").
if: steps.inventory.outputs.found_e2e == 'true' if: steps.inventory.outputs.found_e2e == 'true'
run: | run: |
python scripts/coverage-summary.py \ uv run --project engine --locked --group tools python scripts/coverage-summary.py \
--title "Real user-flow backend coverage (e2e:live + cucumber)" \ --title "Real user-flow backend coverage (e2e:live + cucumber)" \
--jacoco "merged=build/reports/jacoco/aggregate-e2e/jacocoTestReport.xml" \ --jacoco "merged=build/reports/jacoco/aggregate-e2e/jacocoTestReport.xml" \
--github-step-summary --github-step-summary
@@ -169,7 +167,7 @@ jobs:
# is meaningless when one is a strict superset of the other. # is meaningless when one is a strict superset of the other.
if: steps.inventory.outputs.found_all == 'true' if: steps.inventory.outputs.found_all == 'true'
run: | run: |
python scripts/coverage-summary.py \ uv run --project engine --locked --group tools python scripts/coverage-summary.py \
--title "Combined backend coverage (JUnit + e2e:live + cucumber)" \ --title "Combined backend coverage (JUnit + e2e:live + cucumber)" \
--jacoco "merged=build/reports/jacoco/aggregate-all/jacocoTestReport.xml" \ --jacoco "merged=build/reports/jacoco/aggregate-all/jacocoTestReport.xml" \
--github-step-summary --github-step-summary
@@ -224,7 +222,7 @@ jobs:
# generated above) plus whichever frontend artifacts landed. # generated above) plus whichever frontend artifacts landed.
# Every input is optional; missing ones render as "-". # Every input is optional; missing ones render as "-".
run: | run: |
python scripts/coverage-matrix.py \ uv run --project engine --locked --group tools python scripts/coverage-matrix.py \
${{ steps.inventory.outputs.found_all == 'true' && '--jacoco-all build/reports/jacoco/aggregate-all/jacocoTestReport.xml' || '' }} \ ${{ steps.inventory.outputs.found_all == 'true' && '--jacoco-all build/reports/jacoco/aggregate-all/jacocoTestReport.xml' || '' }} \
${{ steps.inventory.outputs.found_e2e == 'true' && '--jacoco-e2e build/reports/jacoco/aggregate-e2e/jacocoTestReport.xml' || '' }} \ ${{ steps.inventory.outputs.found_e2e == 'true' && '--jacoco-e2e build/reports/jacoco/aggregate-e2e/jacocoTestReport.xml' || '' }} \
--vitest matrix-inputs/vitest/coverage-summary.json \ --vitest matrix-inputs/vitest/coverage-summary.json \
.github/workflows/docker-compose-tests.yml
+9 -14
View File
@@ -69,16 +69,17 @@ jobs:
sudo curl -SL "https://github.com/docker/compose/releases/download/v2.39.4/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose sudo curl -SL "https://github.com/docker/compose/releases/download/v2.39.4/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose sudo chmod +x /usr/local/bin/docker-compose
- name: Set up Python - name: Install uv
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with: with:
python-version: "3.12" enable-cache: true
cache: "pip" # caching pip dependencies cache-dependency-glob: |
cache-dependency-path: ./testing/cucumber/requirements.txt engine/pyproject.toml
engine/uv.lock
- name: Pip requirements - name: Install Cucumber and coverage dependencies
run: | run: |
pip install --require-hashes --only-binary=:all: -r ./testing/cucumber/requirements.txt uv sync --project engine --locked --group cucumber --group tools
- name: Extract JaCoCo agent for cucumber coverage - name: Extract JaCoCo agent for cucumber coverage
# Stages build/jacoco/jacocoagent.jar where the coverage override # Stages build/jacoco/jacocoagent.jar where the coverage override
@@ -121,16 +122,10 @@ jobs:
echo "report=false" >> "$GITHUB_OUTPUT" echo "report=false" >> "$GITHUB_OUTPUT"
fi fi
- name: Install defusedxml for coverage summary
# coverage-summary.py parses JaCoCo XML through defusedxml -
# see the script header for context.
if: always() && steps.cucumber-coverage.outputs.report == 'true'
run: python -m pip install --quiet defusedxml
- name: Cucumber coverage step summary - name: Cucumber coverage step summary
if: always() && steps.cucumber-coverage.outputs.report == 'true' if: always() && steps.cucumber-coverage.outputs.report == 'true'
run: | run: |
python scripts/coverage-summary.py \ uv run --project engine --locked --group tools python scripts/coverage-summary.py \
--title "Cucumber (docker) JaCoCo coverage" \ --title "Cucumber (docker) JaCoCo coverage" \
--jacoco "cucumber=build/reports/jacoco/cucumber/jacocoTestReport.xml" \ --jacoco "cucumber=build/reports/jacoco/cucumber/jacocoTestReport.xml" \
--github-step-summary --github-step-summary
.github/workflows/e2e-live.yml
+13 -32
View File
@@ -119,20 +119,18 @@ jobs:
echo "::warning::No e2e:live .exec found at .test-state/playwright/jacoco.exec; skipping report" echo "::warning::No e2e:live .exec found at .test-state/playwright/jacoco.exec; skipping report"
echo "report=false" >> "$GITHUB_OUTPUT" echo "report=false" >> "$GITHUB_OUTPUT"
fi fi
- name: Set up Python for coverage summary - name: Install uv
if: always() && steps.live-coverage.outputs.report == 'true' if: always()
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with: with:
python-version: "3.12" enable-cache: true
- name: Install defusedxml for coverage summary cache-dependency-glob: |
# coverage-summary.py uses defusedxml instead of stdlib xml.etree engine/pyproject.toml
# to dodge XXE / billion-laughs scanner findings. engine/uv.lock
if: always() && steps.live-coverage.outputs.report == 'true'
run: python -m pip install --quiet defusedxml
- name: e2e:live coverage step summary - name: e2e:live coverage step summary
if: always() && steps.live-coverage.outputs.report == 'true' if: always() && steps.live-coverage.outputs.report == 'true'
run: | run: |
python scripts/coverage-summary.py \ uv run --project engine --locked --group tools python scripts/coverage-summary.py \
--title "Playwright (live backend) JaCoCo coverage" \ --title "Playwright (live backend) JaCoCo coverage" \
--jacoco "e2e-live=build/reports/jacoco/e2e-live/jacocoTestReport.xml" \ --jacoco "e2e-live=build/reports/jacoco/e2e-live/jacocoTestReport.xml" \
--github-step-summary --github-step-summary
@@ -155,23 +153,6 @@ jobs:
retention-days: 7 retention-days: 7
if-no-files-found: warn if-no-files-found: warn
- name: Set up Python for frontend coverage summary
# Separate from the backend-coverage python step because the
# frontend path doesn't depend on a JaCoCo report - it produces
# a summary even on backend failure, as long as some Playwright
# tests ran far enough to dump V8 coverage.
if: always()
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.12"
- name: Install defusedxml for frontend coverage summary
# Idempotent re-install: the backend-coverage step may have
# installed it already, but this leg can run on its own when the
# backend report step skips (e.g. .exec missing).
if: always()
run: python -m pip install --quiet defusedxml
- name: Aggregate Playwright frontend (V8) coverage - name: Aggregate Playwright frontend (V8) coverage
# Rolls per-test V8 dumps from the test-base fixture into one # Rolls per-test V8 dumps from the test-base fixture into one
# vitest-shaped coverage-summary.json. Tolerates a missing dump # vitest-shaped coverage-summary.json. Tolerates a missing dump
@@ -182,7 +163,7 @@ jobs:
run: | run: |
if [ -d .test-state/playwright/coverage-pw ] && \ if [ -d .test-state/playwright/coverage-pw ] && \
find .test-state/playwright/coverage-pw -name '*.json' -type f | grep -q .; then find .test-state/playwright/coverage-pw -name '*.json' -type f | grep -q .; then
python scripts/playwright-coverage-summary.py \ uv run --project engine --locked --group tools python scripts/playwright-coverage-summary.py \
.test-state/playwright/coverage-pw \ .test-state/playwright/coverage-pw \
--out .test-state/playwright/coverage-pw-summary/coverage-summary.json --out .test-state/playwright/coverage-pw-summary/coverage-summary.json
echo "summary=true" >> "$GITHUB_OUTPUT" echo "summary=true" >> "$GITHUB_OUTPUT"
@@ -194,10 +175,10 @@ jobs:
- name: Playwright frontend coverage step summary - name: Playwright frontend coverage step summary
if: always() && steps.pw-frontend-coverage.outputs.summary == 'true' if: always() && steps.pw-frontend-coverage.outputs.summary == 'true'
run: | run: |
python scripts/coverage-summary.py \ uv run --project engine --locked --group tools python scripts/coverage-summary.py \
--title "Playwright (live) frontend coverage" \ --title "Playwright (live) frontend coverage" \
--vitest .test-state/playwright/coverage-pw-summary/coverage-summary.json \ --vitest .test-state/playwright/coverage-pw-summary/coverage-summary.json \
--github-step-summary --github-step-summary
- name: Upload Playwright frontend coverage - name: Upload Playwright frontend coverage
# Bundle both the aggregated summary and the raw V8 dumps so # Bundle both the aggregated summary and the raw V8 dumps so
.github/workflows/frontend-validation.yml
+7 -9
View File
@@ -115,20 +115,18 @@ jobs:
id: frontend-coverage id: frontend-coverage
continue-on-error: true continue-on-error: true
run: task frontend:test:coverage run: task frontend:test:coverage
- name: Set up Python for coverage summary - name: Install uv
if: always() if: always()
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with: with:
python-version: "3.12" enable-cache: true
- name: Install defusedxml for coverage summary cache-dependency-glob: |
# See coverage-summary.py header - it parses XML through defusedxml engine/pyproject.toml
# to dodge the stdlib parser's exposure to XXE / billion-laughs. engine/uv.lock
if: always()
run: python -m pip install --quiet defusedxml
- name: Vitest coverage step summary - name: Vitest coverage step summary
if: always() if: always()
run: | run: |
python scripts/coverage-summary.py \ uv run --project engine --locked --group tools python scripts/coverage-summary.py \
--title "Frontend Vitest coverage" \ --title "Frontend Vitest coverage" \
--vitest frontend/editor/coverage/coverage-summary.json \ --vitest frontend/editor/coverage/coverage-summary.json \
--github-step-summary --github-step-summary
.github/workflows/multiOSReleases.yml
+8 -2
View File
@@ -887,10 +887,16 @@ jobs:
# Gate publish on valid updater sigs. Runs after the review upload (so # Gate publish on valid updater sigs. Runs after the review upload (so
# artifacts survive for debugging) and before action-gh-release. # artifacts survive for debugging) and before action-gh-release.
- name: Install uv
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with:
enable-cache: true
cache-dependency-glob: |
engine/pyproject.toml
engine/uv.lock
- name: Verify updater signatures - name: Verify updater signatures
run: | run: |
python3 -m pip install --quiet 'cryptography==44.0.0' uv run --project engine --locked --only-group updater-signatures python .github/scripts/verify-updater-signatures.py \
python3 .github/scripts/verify-updater-signatures.py \
./artifacts/tauri frontend/editor/src-tauri/tauri.conf.json ./artifacts/tauri frontend/editor/src-tauri/tauri.conf.json
# workflow_dispatch path requires platform=='all' so a single-platform # workflow_dispatch path requires platform=='all' so a single-platform
.github/workflows/pre_commit.yml
+3
View File
@@ -28,6 +28,9 @@ jobs:
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0 uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with: with:
enable-cache: true enable-cache: true
cache-dependency-glob: |
engine/pyproject.toml
engine/uv.lock
cache-suffix: pre-commit cache-suffix: pre-commit
- name: Install Task - name: Install Task
.github/workflows/sync_files_v2.yml
+9 -12
View File
@@ -51,28 +51,25 @@ jobs:
app-id: ${{ secrets.GH_APP_ID }} app-id: ${{ secrets.GH_APP_ID }}
private-key: ${{ secrets.GH_APP_PRIVATE_KEY }} private-key: ${{ secrets.GH_APP_PRIVATE_KEY }}
- name: Set up Python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.12"
cache: "pip" # caching pip dependencies
- name: Install Python dependencies
run: |
pip install --require-hashes --only-binary=:all: -r ./.github/scripts/requirements_sync_readme.txt
- name: Install uv - name: Install uv
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0 uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with: with:
enable-cache: true enable-cache: true
cache-dependency-glob: |
engine/pyproject.toml
engine/uv.lock
cache-suffix: sync-files cache-suffix: sync-files
- name: Install Python dependencies
run: |
uv sync --project engine --locked --group tools
- name: Install Task - name: Install Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0 uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
- name: Sync translation TOML files - name: Sync translation TOML files
run: | run: |
python .github/scripts/check_language_toml.py --reference-file "frontend/editor/public/locales/en-US/translation.toml" --branch main uv run --project engine --locked --group tools python .github/scripts/check_language_toml.py --reference-file "frontend/editor/public/locales/en-US/translation.toml" --branch main
- name: Sort translation TOML files - name: Sort translation TOML files
run: | run: |
@@ -85,7 +82,7 @@ jobs:
- name: Sync README.md - name: Sync README.md
run: | run: |
python scripts/counter_translation_v3.py uv run --project engine --locked --group tools python scripts/counter_translation_v3.py
- name: Run git add - name: Run git add
run: | run: |
.taskfiles/engine.yml
+40 -13
View File
@@ -2,22 +2,49 @@ version: '3'
tasks: tasks:
install: install:
desc: "Install engine dependencies" desc: "Install engine runtime and development dependencies"
run: once run: once
cmds: cmds:
- uv python install 3.13.8 - uv python install 3.13.8
- uv sync - uv sync --locked --group engine --group engine-dev
sources: sources:
- uv.lock - uv.lock
- pyproject.toml - pyproject.toml
status: status:
- test -d .venv - test -d .venv
lock:
desc: "Update the engine lockfile from project metadata"
cmds:
- uv lock
lock:upgrade:
desc: "Upgrade allowed engine dependencies and update the lockfile"
cmds:
- uv lock --upgrade
lock:check:
desc: "Check whether the engine lockfile is current"
cmds:
- uv lock --check
update:
desc: "Upgrade engine dependencies and synchronize the environment"
cmds:
- task: lock:upgrade
- uv sync --locked --group engine --group engine-dev
update:all:
desc: "Upgrade all Python dependency groups and synchronize the environment"
cmds:
- task: lock:upgrade
- uv sync --locked --all-groups
prepare: prepare:
desc: "Set up engine .env from template" desc: "Set up engine .env from template"
deps: [install] deps: [install]
cmds: cmds:
- uv run scripts/setup_env.py - uv run --locked --group engine --group engine-dev scripts/setup_env.py
sources: sources:
- scripts/setup_env.py - scripts/setup_env.py
generates: generates:
@@ -33,7 +60,7 @@ tasks:
env: env:
PYTHONUNBUFFERED: "1" PYTHONUNBUFFERED: "1"
cmds: cmds:
- uv run uvicorn stirling.api.app:app --host 0.0.0.0 --port {{.PORT}} --workers "${STIRLING_ENGINE_WORKERS:-4}" - uv run --locked --group engine uvicorn stirling.api.app:app --host 0.0.0.0 --port {{.PORT}} --workers "${STIRLING_ENGINE_WORKERS:-4}"
dev: dev:
desc: "Start engine dev server with hot reload" desc: "Start engine dev server with hot reload"
@@ -45,43 +72,43 @@ tasks:
env: env:
PYTHONUNBUFFERED: "1" PYTHONUNBUFFERED: "1"
cmds: cmds:
- uv run uvicorn stirling.api.app:app --host 0.0.0.0 --port {{.PORT}} --reload - uv run --locked --group engine --group engine-dev uvicorn stirling.api.app:app --host 0.0.0.0 --port {{.PORT}} --reload
lint: lint:
desc: "Run linting" desc: "Run linting"
deps: [install] deps: [install]
cmds: cmds:
- uv run ruff check . - uv run --locked --group engine --group engine-dev ruff check .
lint:fix: lint:fix:
desc: "Auto-fix lint issues" desc: "Auto-fix lint issues"
deps: [install] deps: [install]
cmds: cmds:
- uv run ruff check . --fix - uv run --locked --group engine --group engine-dev ruff check . --fix
format: format:
desc: "Auto-fix code formatting" desc: "Auto-fix code formatting"
deps: [install] deps: [install]
cmds: cmds:
- uv run ruff format . - uv run --locked --group engine --group engine-dev ruff format .
format:check: format:check:
desc: "Check code formatting" desc: "Check code formatting"
deps: [install] deps: [install]
cmds: cmds:
- uv run ruff format . --diff - uv run --locked --group engine --group engine-dev ruff format . --diff
typecheck: typecheck:
desc: "Run type checking" desc: "Run type checking"
deps: [install] deps: [install]
cmds: cmds:
- uv run pyright . --warnings - uv run --locked --group engine --group engine-dev pyright . --warnings
test: test:
desc: "Run tests" desc: "Run tests"
deps: [prepare] deps: [prepare]
cmds: cmds:
- uv run pytest tests - uv run --locked --group engine --group engine-dev pytest tests
fix: fix:
desc: "Auto-fix lint + format" desc: "Auto-fix lint + format"
@@ -102,7 +129,7 @@ tasks:
desc: "Generate tool_models.py from Java OpenAPI spec (SwaggerDoc.json)" desc: "Generate tool_models.py from Java OpenAPI spec (SwaggerDoc.json)"
deps: [install, ":backend:swagger"] deps: [install, ":backend:swagger"]
cmds: cmds:
- uv run python scripts/generate_tool_models.py --spec ../SwaggerDoc.json --output src/stirling/models/tool_models.py --io-output src/stirling/models/tool_io.py - uv run --locked --group engine --group engine-dev python scripts/generate_tool_models.py --spec ../SwaggerDoc.json --output src/stirling/models/tool_models.py --io-output src/stirling/models/tool_io.py
sources: sources:
- ../SwaggerDoc.json - ../SwaggerDoc.json
- scripts/generate_tool_models.py - scripts/generate_tool_models.py
@@ -114,7 +141,7 @@ tasks:
desc: "Fail if the committed tool models are out of date" desc: "Fail if the committed tool models are out of date"
deps: [install, ":backend:swagger"] deps: [install, ":backend:swagger"]
cmds: cmds:
- uv run python scripts/generate_tool_models.py --spec ../SwaggerDoc.json --output src/stirling/models/tool_models.py --io-output src/stirling/models/tool_io.py --check - uv run --locked --group engine --group engine-dev python scripts/generate_tool_models.py --spec ../SwaggerDoc.json --output src/stirling/models/tool_models.py --io-output src/stirling/models/tool_io.py --check
clean: clean:
desc: "Clean build artifacts" desc: "Clean build artifacts"
.taskfiles/pre-commit.yml
+17 -13
View File
@@ -45,6 +45,10 @@ vars:
# which owns the version and caches the binary here. # which owns the version and caches the binary here.
GITLEAKS_BIN: '.task/bin/gitleaks{{if eq OS "windows"}}.exe{{end}}' GITLEAKS_BIN: '.task/bin/gitleaks{{if eq OS "windows"}}.exe{{end}}'
env:
# Keep repository-wide checks isolated from the engine runtime environment.
UV_PROJECT_ENVIRONMENT: '.venv-pre-commit'
tasks: tasks:
default: default:
desc: "Check formatting, spelling, and secrets across the repo" desc: "Check formatting, spelling, and secrets across the repo"
@@ -76,19 +80,19 @@ tasks:
desc: "Install the pinned pre-commit Python tools" desc: "Install the pinned pre-commit Python tools"
run: once run: once
cmds: cmds:
- uv sync --project scripts/pre-commit --locked - uv sync --project engine --locked --group pre-commit
sources: sources:
- scripts/pre-commit/uv.lock - engine/uv.lock
- scripts/pre-commit/pyproject.toml - engine/pyproject.toml
status: status:
- test -d scripts/pre-commit/.venv - test -d engine/.venv-pre-commit
clean: clean:
desc: "Remove the cached gitleaks binary and the tool virtualenv" desc: "Remove the cached gitleaks binary and the pre-commit virtualenv"
cmds: cmds:
- cmd: rm -rf scripts/pre-commit/.venv .task/bin/gitleaks - cmd: rm -rf engine/.venv-pre-commit .task/bin/gitleaks
platforms: [linux, darwin] platforms: [linux, darwin]
- cmd: cmd /c "rmdir /s /q scripts\pre-commit\.venv & del /q .task\bin\gitleaks.exe" - cmd: cmd /c "rmdir /s /q engine\.venv-pre-commit & del /q .task\bin\gitleaks.exe"
platforms: [windows] platforms: [windows]
ignore_error: true ignore_error: true
@@ -97,26 +101,26 @@ tasks:
ruff: ruff:
deps: [install] deps: [install]
cmds: cmds:
- uv run --project scripts/pre-commit --no-sync ruff check --line-length=127 {{if .FIX}}--fix {{end}}$(git ls-files {{.PY_FILES}}) - uv run --project engine --locked --group pre-commit ruff check --isolated --line-length=120 {{if .FIX}}--fix {{end}}$(git ls-files {{.PY_FILES}})
ruff-format: ruff-format:
deps: [install] deps: [install]
cmds: cmds:
- uv run --project scripts/pre-commit --no-sync ruff format {{if .FIX}}{{else}}--check {{end}}$(git ls-files {{.PY_FILES}}) - uv run --project engine --locked --group pre-commit ruff format --isolated --line-length=120 {{if .FIX}}{{else}}--check {{end}}$(git ls-files {{.PY_FILES}})
codespell: codespell:
deps: [install] deps: [install]
cmds: cmds:
- uv run --project scripts/pre-commit --no-sync codespell --ignore-words-list=thirdParty,tabEl,tabEls,Sie,ist,fulfilment --quiet-level=2 $(git ls-files {{.SPELL_FILES}}) - uv run --project engine --locked --group pre-commit codespell --ignore-words-list=thirdParty,tabEl,tabEls,Sie,ist,fulfilment --quiet-level=2 $(git ls-files {{.SPELL_FILES}})
toml-sort: toml-sort:
deps: [install] deps: [install]
cmds: cmds:
- uv run --project scripts/pre-commit --no-sync python scripts/pre-commit/sort_locale_toml.py {{if .FIX}}--fix {{end}}{{.LOCALE_TOML}} - uv run --project engine --locked --group pre-commit python scripts/pre-commit/sort_locale_toml.py {{if .FIX}}--fix {{end}}{{.LOCALE_TOML}}
whitespace: whitespace:
cmds: cmds:
- uv run --no-project python scripts/pre-commit/whitespace.py {{if .FIX}}--fix {{end}}{{.WS_FILES}} - uv run --project engine --locked --group pre-commit python scripts/pre-commit/whitespace.py {{if .FIX}}--fix {{end}}{{.WS_FILES}}
gitleaks: gitleaks:
deps: [gitleaks-bin] deps: [gitleaks-bin]
@@ -130,4 +134,4 @@ tasks:
internal: true internal: true
desc: "Ensure the pinned, checksum-verified gitleaks binary is cached in .task/bin" desc: "Ensure the pinned, checksum-verified gitleaks binary is cached in .task/bin"
cmds: cmds:
- uv run --no-project python scripts/pre-commit/install_gitleaks.py - uv run --project engine --locked --group pre-commit python scripts/pre-commit/install_gitleaks.py
.vscode/extensions.json
Vendored
+1 -2
View File
@@ -2,8 +2,6 @@
"recommendations": [ "recommendations": [
"elagil.pre-commit-helper", // Support for pre-commit hooks to enforce code quality "elagil.pre-commit-helper", // Support for pre-commit hooks to enforce code quality
"josevseb.google-java-format-for-vs-code", // Google Java code formatter to follow the Google Java Style Guide "josevseb.google-java-format-for-vs-code", // Google Java code formatter to follow the Google Java Style Guide
"ms-python.black-formatter", // Python code formatter using Black
"ms-python.flake8", // Flake8 linter for Python to enforce code quality
"ms-python.python", // Official Microsoft Python extension with IntelliSense, debugging, and Jupyter support "ms-python.python", // Official Microsoft Python extension with IntelliSense, debugging, and Jupyter support
"ms-vscode-remote.vscode-remote-extensionpack", // Remote Development Pack for SSH, WSL, and Containers "ms-vscode-remote.vscode-remote-extensionpack", // Remote Development Pack for SSH, WSL, and Containers
// "Oracle.oracle-java", // Oracle Java extension with additional features for Java development // "Oracle.oracle-java", // Oracle Java extension with additional features for Java development
@@ -19,6 +17,7 @@
"yzhang.markdown-all-in-one", // Markdown All-in-One extension for enhanced Markdown editing "yzhang.markdown-all-in-one", // Markdown All-in-One extension for enhanced Markdown editing
"stylelint.vscode-stylelint", // Stylelint extension for CSS and SCSS linting "stylelint.vscode-stylelint", // Stylelint extension for CSS and SCSS linting
"redhat.vscode-yaml", // YAML extension for Visual Studio Code "redhat.vscode-yaml", // YAML extension for Visual Studio Code
"tamasfe.even-better-toml", // TOML language support and formatting
"oxc.oxc-vscode", // Oxc (oxlint) extension for JavaScript/TypeScript linting "oxc.oxc-vscode", // Oxc (oxlint) extension for JavaScript/TypeScript linting
] ]
} }
.vscode/settings.json
Vendored
+2 -1
View File
@@ -20,8 +20,9 @@
"editor.defaultFormatter": "vscode.json-language-features" "editor.defaultFormatter": "vscode.json-language-features"
}, },
"[python]": { "[python]": {
"editor.defaultFormatter": "ms-python.black-formatter" "editor.defaultFormatter": "charliermarsh.ruff"
}, },
"ruff.configuration": "${workspaceFolder}/engine/pyproject.toml",
"[gradle-kotlin-dsl]": { "[gradle-kotlin-dsl]": {
"editor.defaultFormatter": "vscjava.vscode-gradle" "editor.defaultFormatter": "vscjava.vscode-gradle"
}, },
app/core/src/main/resources/static/python/png_to_webp.py
+2 -3
View File
@@ -16,6 +16,7 @@ To adjust the DPI resolution for rendering PDF pages:
import argparse import argparse
import os import os
from pdf2image import convert_from_path from pdf2image import convert_from_path
from PIL import Image from PIL import Image
@@ -149,9 +150,7 @@ def main(pdf_image_path, output_dir, dpi=300, single_images_flag=False):
if __name__ == "__main__": if __name__ == "__main__":
parser = argparse.ArgumentParser(description="Convert a PDF file to WebP images.") parser = argparse.ArgumentParser(description="Convert a PDF file to WebP images.")
parser.add_argument("pdf_path", help="The path to the input PDF file.") parser.add_argument("pdf_path", help="The path to the input PDF file.")
parser.add_argument( parser.add_argument("output_dir", help="The directory where the WebP images should be saved.")
"output_dir", help="The directory where the WebP images should be saved."
)
parser.add_argument( parser.add_argument(
"--dpi", "--dpi",
type=int, type=int,
engine/Dockerfile
+4 -6
View File
@@ -12,14 +12,12 @@ RUN apt-get update \
# Source under /app/engine/ to match root Taskfile's `includes.engine.dir: engine`. # Source under /app/engine/ to match root Taskfile's `includes.engine.dir: engine`.
WORKDIR /app/engine WORKDIR /app/engine
COPY engine/pyproject.toml engine/uv.lock engine/.env ./
COPY pyproject.toml uv.lock .env ./ COPY engine/scripts/ ./scripts/
COPY scripts/ ./scripts/
RUN --mount=type=cache,target=/root/.cache/uv \ RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --frozen --no-dev uv sync --frozen --no-dev --group engine
COPY src/ ./src/
COPY engine/src/ ./src/
WORKDIR /app WORKDIR /app
COPY Taskfile.yml ./ COPY Taskfile.yml ./
COPY .taskfiles/ ./.taskfiles/ COPY .taskfiles/ ./.taskfiles/
engine/Dockerfile.dev
+4 -4
View File
@@ -2,13 +2,13 @@
FROM ghcr.io/astral-sh/uv:python3.13-bookworm-slim@sha256:531f855bda2c73cd6ef67d56b733b357cea384185b3022bd09f05e002cd144ca FROM ghcr.io/astral-sh/uv:python3.13-bookworm-slim@sha256:531f855bda2c73cd6ef67d56b733b357cea384185b3022bd09f05e002cd144ca
WORKDIR /app WORKDIR /app
COPY pyproject.toml uv.lock ./ COPY engine/pyproject.toml engine/uv.lock ./engine/
RUN --mount=type=cache,target=/root/.cache/uv \ RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --frozen cd engine && uv sync --frozen --group engine --group engine-dev
ENV PATH="/app/.venv/bin:$PATH" ENV PATH="/app/engine/.venv/bin:$PATH"
ENV PYTHONUNBUFFERED=1 ENV PYTHONUNBUFFERED=1
EXPOSE 5001 EXPOSE 5001
CMD ["uv", "run", "uvicorn", "stirling.api.app:app", "--host", "0.0.0.0", "--port", "5001", "--reload"] CMD ["uv", "run", "--project", "/app/engine", "--frozen", "--group", "engine", "uvicorn", "stirling.api.app:app", "--host", "0.0.0.0", "--port", "5001", "--reload"]
engine/pyproject.toml
+74 -32
View File
@@ -1,33 +1,72 @@
[project] [project]
name = "engine" name = "engine"
version = "0.1.0" version = "0.1.0"
description = "AI Document Engine" description = "Stirling PDF Python projects"
requires-python = ">=3.13" requires-python = ">=3.13,<3.14"
dependencies = [ dependencies = []
"cryptography>=50.0.0",
"fastapi>=0.116.0",
"pgvector>=0.3.6",
"psycopg[binary,pool]>=3.2",
"pydantic>=2.0.0",
# <2 cap: 1.99.0 patches CVE-2026-46678; 2.0 is an untested major migration
"pydantic-ai>=1.99.0,<2.0.0",
"pydantic-ai-slim[voyageai]>=1.99.0,<2.0.0",
"pydantic-settings>=2.0.0",
"python-dotenv>=1.2.1",
"sqlite-vec>=0.1.6",
"uvicorn>=0.35.0",
"opentelemetry-sdk>=1.39.0",
"posthog>=3.0.0",
]
[dependency-groups] [dependency-groups]
dev = [ # Runtime dependencies for the AI document engine service.
"anyio>=4.0.0", engine = [
"datamodel-code-generator[ruff]>=0.26.0", "cryptography>=50.0.0",
"pytest>=8.0.0", "fastapi>=0.141.1",
"pyright>=1.1.408", "opentelemetry-sdk>=1.39.1",
"referencing>=0.35.0", "pgvector>=0.5.0",
"ruff>=0.14.10", "posthog>=7.38.3",
"psycopg[binary,pool]>=3.3.4",
"pydantic>=2.13.4",
# <2 cap: 1.99.0 patches CVE-2026-46678; 2.0 is an untested major migration.
"pydantic-ai>=1.107.2,<2.0.0",
"pydantic-ai-slim[voyageai]>=1.107.2,<2.0.0",
"pydantic-settings>=2.15.0",
"python-dotenv>=1.2.2",
"sqlite-vec>=0.1.9",
"uvicorn>=0.52.1",
]
# Type checking, testing, model generation, and formatting tools for the engine.
engine-dev = [
"anyio>=4.14.2",
"datamodel-code-generator[ruff]==0.64.0",
"pyright>=1.1.411",
"pytest>=9.1.1",
"referencing>=0.37.0",
"ruff==0.15.5",
]
# Dependencies for the Cucumber/Python integration test suite.
cucumber = [
"behave>=1.3.3",
"behave-html-formatter>=0.9.10",
"opencv-python-headless>=5.0.0.93",
"pdf2image>=1.17.0",
"pillow>=12.3.0",
"pypdf[crypto]>=6.15.0",
"qrcode[pil]>=8.2",
"reportlab>=5.0.0",
"requests>=2.34.2",
]
# Shared Python utilities used by repository scripts and CI workflows.
tools = [
"deep-translator>=1.11.4",
"defusedxml>=0.7.1",
"fonttools>=4.63.0",
"fpdf2>=2.8.7",
"openai>=2.53.0",
"requests>=2.34.2",
"tomli-w>=1.2.0",
"tomlkit>=0.15.1",
"urllib3>=2.7.0",
"unoserver>=3.7",
"weasyprint>=69.0",
]
# Release-signature verification dependencies.
updater-signatures = [
"cryptography>=50.0.0",
]
# Pinned repository-wide pre-commit tooling.
pre-commit = [
"codespell==2.4.2",
"ruff==0.15.5",
"tomli-w==1.2.0",
] ]
[build-system] [build-system]
@@ -36,9 +75,10 @@ build-backend = "hatchling.build"
[tool.hatch.build.targets.wheel] [tool.hatch.build.targets.wheel]
packages = ["src"] packages = ["src"]
exclude = [ exclude = ["tests"]
"tests",
] [tool.uv]
default-groups = []
[tool.ruff] [tool.ruff]
line-length = 120 line-length = 120
@@ -53,14 +93,16 @@ select = [
"W", "W",
"RUF100", "RUF100",
"UP", "UP",
"PYI", # flake8-pyi: flags deprecated typing constructs "PYI", # flake8-pyi: flags deprecated typing constructs
"FA", # flake8-future-annotations: flags missing future annotations imports "FA", # flake8-future-annotations: flags missing future annotations imports
"BLE", # flake8-blind-except: flags bare `except Exception` "BLE", # flake8-blind-except: flags bare `except Exception`
] ]
[tool.ruff.lint.isort]
known-first-party = ["stirling", "tests"]
[tool.pyright] [tool.pyright]
pythonVersion = "3.13" pythonVersion = "3.13"
reportImportCycles = "warning" reportImportCycles = "warning"
reportUnnecessaryCast = "warning" reportUnnecessaryCast = "warning"
reportUnnecessaryTypeIgnoreComment = "warning" reportUnnecessaryTypeIgnoreComment = "warning"
engine/src/stirling/services/runtime.py
+2 -2
View File
@@ -4,7 +4,7 @@ import asyncio
import json import json
import logging import logging
import os import os
from collections.abc import AsyncIterator from collections.abc import AsyncGenerator
from contextlib import asynccontextmanager from contextlib import asynccontextmanager
from dataclasses import dataclass from dataclasses import dataclass
from typing import Any, assert_never from typing import Any, assert_never
@@ -133,7 +133,7 @@ class ConcurrencyLimitedModel(WrapperModel):
model_settings: ModelSettings | None, model_settings: ModelSettings | None,
model_request_parameters: ModelRequestParameters, model_request_parameters: ModelRequestParameters,
run_context: RunContext[Any] | None = None, run_context: RunContext[Any] | None = None,
) -> AsyncIterator[StreamedResponse]: ) -> AsyncGenerator[StreamedResponse]:
async with self._semaphore: async with self._semaphore:
async with super().request_stream( async with super().request_stream(
messages, model_settings, model_request_parameters, run_context messages, model_settings, model_request_parameters, run_context
engine/tests/test_config_routes.py
+2 -2
View File
@@ -4,7 +4,7 @@ from __future__ import annotations
import asyncio import asyncio
import threading import threading
from collections.abc import Callable, Iterator from collections.abc import Callable, Generator
from contextlib import contextmanager from contextlib import contextmanager
from pathlib import Path from pathlib import Path
from unittest.mock import patch from unittest.mock import patch
@@ -31,7 +31,7 @@ def _client(
settings_factory: Callable[[], AppSettings], settings_factory: Callable[[], AppSettings],
*, *,
client_addr: tuple[str, int] = ("127.0.0.1", 12345), client_addr: tuple[str, int] = ("127.0.0.1", 12345),
) -> Iterator[TestClient]: ) -> Generator[TestClient]:
"""Enter a TestClient whose lifespan builds app.state from ``settings_factory``.""" """Enter a TestClient whose lifespan builds app.state from ``settings_factory``."""
previous = app.dependency_overrides.get(load_settings) previous = app.dependency_overrides.get(load_settings)
app.dependency_overrides[load_settings] = settings_factory app.dependency_overrides[load_settings] = settings_factory
engine/uv.lock
Generated
+1099 -1348
View File
File diff suppressed because it is too large Load Diff
scripts/analyze_pdf_json.py
+12 -22
View File
@@ -15,9 +15,10 @@ from __future__ import annotations
import argparse import argparse
import json import json
import math import math
from pathlib import Path from collections.abc import Iterable
from dataclasses import dataclass from dataclasses import dataclass
from typing import Any, Dict, Iterable, List, Tuple from pathlib import Path
from typing import Any
def human_bytes(value: float) -> str: def human_bytes(value: float) -> str:
@@ -49,7 +50,7 @@ class FontBreakdown:
web_program_bytes: int = 0 web_program_bytes: int = 0
pdf_program_bytes: int = 0 pdf_program_bytes: int = 0
metadata_bytes: int = 0 metadata_bytes: int = 0
sample_cos_ids: List[Tuple[str | None, str | None]] = None sample_cos_ids: list[tuple[str | None, str | None]] = None
@dataclass @dataclass
@@ -82,7 +83,7 @@ def approx_struct_size(obj: Any) -> int:
return len(json.dumps(obj, separators=(",", ":"))) return len(json.dumps(obj, separators=(",", ":")))
def analyze_fonts(fonts: Iterable[Dict[str, Any]]) -> FontBreakdown: def analyze_fonts(fonts: Iterable[dict[str, Any]]) -> FontBreakdown:
total = 0 total = 0
with_cos = 0 with_cos = 0
with_prog = 0 with_prog = 0
@@ -92,7 +93,7 @@ def analyze_fonts(fonts: Iterable[Dict[str, Any]]) -> FontBreakdown:
web_program_bytes = 0 web_program_bytes = 0
pdf_program_bytes = 0 pdf_program_bytes = 0
metadata_bytes = 0 metadata_bytes = 0
sample_cos_ids: List[Tuple[str | None, str | None]] = [] sample_cos_ids: list[tuple[str | None, str | None]] = []
for font in fonts: for font in fonts:
total += 1 total += 1
@@ -105,11 +106,7 @@ def analyze_fonts(fonts: Iterable[Dict[str, Any]]) -> FontBreakdown:
sample_cos_ids.append((font_id, uid)) sample_cos_ids.append((font_id, uid))
metadata_bytes += approx_struct_size( metadata_bytes += approx_struct_size(
{ {k: v for k, v in font.items() if k not in {"program", "webProgram", "pdfProgram"}}
k: v
for k, v in font.items()
if k not in {"program", "webProgram", "pdfProgram"}
}
) )
program = font.get("program") program = font.get("program")
@@ -140,7 +137,7 @@ def analyze_fonts(fonts: Iterable[Dict[str, Any]]) -> FontBreakdown:
) )
def analyze_pages(pages: Iterable[Dict[str, Any]]) -> PageBreakdown: def analyze_pages(pages: Iterable[dict[str, Any]]) -> PageBreakdown:
page_count = 0 page_count = 0
total_text = 0 total_text = 0
total_images = 0 total_images = 0
@@ -185,7 +182,7 @@ def analyze_pages(pages: Iterable[Dict[str, Any]]) -> PageBreakdown:
) )
def analyze_document(document: Dict[str, Any], total_size: int) -> DocumentBreakdown: def analyze_document(document: dict[str, Any], total_size: int) -> DocumentBreakdown:
fonts = document.get("fonts") or [] fonts = document.get("fonts") or []
pages = document.get("pages") or [] pages = document.get("pages") or []
metadata = document.get("metadata") or {} metadata = document.get("metadata") or {}
@@ -259,17 +256,10 @@ def main() -> None:
print(f" XMP metadata bytes: {human_bytes(summary.xmp_bytes)}") print(f" XMP metadata bytes: {human_bytes(summary.xmp_bytes)}")
print(f" Form fields bytes: {human_bytes(summary.form_fields_bytes)}") print(f" Form fields bytes: {human_bytes(summary.form_fields_bytes)}")
print(f" Lazy flag bytes: {summary.lazy_flag_bytes}") print(f" Lazy flag bytes: {summary.lazy_flag_bytes}")
print( print(f" Text payload characters (not counting JSON overhead): {page_stats.text_payload_chars:,}")
f" Text payload characters (not counting JSON overhead): "
f"{page_stats.text_payload_chars:,}"
)
print(f" Approx text structure bytes: {human_bytes(page_stats.text_struct_bytes)}") print(f" Approx text structure bytes: {human_bytes(page_stats.text_struct_bytes)}")
print( print(f" Approx image structure bytes: {human_bytes(page_stats.image_struct_bytes)}")
f" Approx image structure bytes: {human_bytes(page_stats.image_struct_bytes)}" print(f" Approx content stream bytes: {human_bytes(page_stats.content_stream_bytes)}")
)
print(
f" Approx content stream bytes: {human_bytes(page_stats.content_stream_bytes)}"
)
print(f" Approx annotations bytes: {human_bytes(page_stats.annotations_bytes)}") print(f" Approx annotations bytes: {human_bytes(page_stats.annotations_bytes)}")
scripts/convert_cff_to_ttf.py
+11 -24
View File
@@ -4,12 +4,13 @@ Wrap raw CFF/Type1C data (extracted from PDFs) as OpenType-CFF for web compatibi
Builds proper Unicode cmap from PDF ToUnicode data. Builds proper Unicode cmap from PDF ToUnicode data.
""" """
import sys
import re import re
from pathlib import Path import sys
from io import BytesIO from io import BytesIO
from fontTools.ttLib import TTFont, newTable from pathlib import Path
from fontTools.cffLib import CFFFontSet from fontTools.cffLib import CFFFontSet
from fontTools.ttLib import TTFont, newTable
from fontTools.ttLib.tables._c_m_a_p import cmap_format_4, cmap_format_12 from fontTools.ttLib.tables._c_m_a_p import cmap_format_4, cmap_format_12
from fontTools.ttLib.tables._n_a_m_e import NameRecord from fontTools.ttLib.tables._n_a_m_e import NameRecord
from fontTools.ttLib.tables.O_S_2f_2 import Panose from fontTools.ttLib.tables.O_S_2f_2 import Panose
@@ -117,15 +118,11 @@ def wrap_cff_as_otf(input_path, output_path, tounicode_path=None):
# Get glyph names # Get glyph names
if hasattr(cff_font, "charset") and cff_font.charset is not None: if hasattr(cff_font, "charset") and cff_font.charset is not None:
glyph_order = [".notdef"] + [ glyph_order = [".notdef"] + [name for name in cff_font.charset if name != ".notdef"]
name for name in cff_font.charset if name != ".notdef"
]
else: else:
# Fallback to CharStrings keys # Fallback to CharStrings keys
charstrings = cff_font.CharStrings charstrings = cff_font.CharStrings
glyph_order = [".notdef"] + [ glyph_order = [".notdef"] + [name for name in charstrings.keys() if name != ".notdef"]
name for name in charstrings.keys() if name != ".notdef"
]
otf.setGlyphOrder(glyph_order) otf.setGlyphOrder(glyph_order)
@@ -139,9 +136,7 @@ def wrap_cff_as_otf(input_path, output_path, tounicode_path=None):
# Get defaults from CFF Private dict # Get defaults from CFF Private dict
private_dict = getattr(cff_font, "Private", None) private_dict = getattr(cff_font, "Private", None)
default_width = ( default_width = getattr(private_dict, "defaultWidthX", 500) if private_dict else 500
getattr(private_dict, "defaultWidthX", 500) if private_dict else 500
)
# Calculate bounding box, widths, and LSBs # Calculate bounding box, widths, and LSBs
x_min = 0 x_min = 0
@@ -280,9 +275,7 @@ def wrap_cff_as_otf(input_path, output_path, tounicode_path=None):
# For CID fonts: glyph names are "cid00123" (5-digit zero-padded) # For CID fonts: glyph names are "cid00123" (5-digit zero-padded)
# For non-CID fonts: glyph names vary but GID == array index # For non-CID fonts: glyph names vary but GID == array index
is_cid_font = any( is_cid_font = any(gn.startswith("cid") for gn in glyph_order[1:6]) # Check first few non-.notdef glyphs
gn.startswith("cid") for gn in glyph_order[1:6]
) # Check first few non-.notdef glyphs
for gid, unicode_val in gid_to_unicode.items(): for gid, unicode_val in gid_to_unicode.items():
if unicode_val > 0: if unicode_val > 0:
@@ -355,14 +348,10 @@ def wrap_cff_as_otf(input_path, output_path, tounicode_path=None):
cmap4_mac.cmap = {cp: gn for cp, gn in unicode_to_glyph.items() if cp <= 0xFFFF} cmap4_mac.cmap = {cp: gn for cp, gn in unicode_to_glyph.items() if cp <= 0xFFFF}
cmap_tables.append(cmap4_mac) cmap_tables.append(cmap4_mac)
cmap.tables = [t for t in cmap_tables if t.cmap] or [ cmap.tables = [t for t in cmap_tables if t.cmap] or [cmap4_win] # Ensure at least one
cmap4_win
] # Ensure at least one
otf["cmap"] = cmap otf["cmap"] = cmap
print( print(f"Built cmap with {len(unicode_to_glyph)} Unicode mappings", file=sys.stderr)
f"Built cmap with {len(unicode_to_glyph)} Unicode mappings", file=sys.stderr
)
# === Create OS/2 table with correct metrics === # === Create OS/2 table with correct metrics ===
os2 = newTable("OS/2") os2 = newTable("OS/2")
@@ -515,9 +504,7 @@ Examples:
# Add named arguments # Add named arguments
parser.add_argument("--input", dest="input_file", help="Input CFF file path") parser.add_argument("--input", dest="input_file", help="Input CFF file path")
parser.add_argument("--output", dest="output_file", help="Output OTF file path") parser.add_argument("--output", dest="output_file", help="Output OTF file path")
parser.add_argument( parser.add_argument("--to-unicode", dest="tounicode_file", help="ToUnicode mapping file path")
"--to-unicode", dest="tounicode_file", help="ToUnicode mapping file path"
)
# Add positional arguments for backward compatibility # Add positional arguments for backward compatibility
parser.add_argument("input_pos", nargs="?", help="Input CFF file (positional)") parser.add_argument("input_pos", nargs="?", help="Input CFF file (positional)")
scripts/counter_translation_v3.py
+6 -18
View File
@@ -50,16 +50,13 @@ import glob
import os import os
import re import re
import sys import sys
from collections.abc import Mapping from collections.abc import Iterable, Mapping
from typing import Iterable
# Ensure tomlkit is installed before importing # Ensure tomlkit is installed before importing
try: try:
import tomlkit import tomlkit
except ImportError: except ImportError:
raise ImportError( raise ImportError("The 'tomlkit' library is not installed. Please install it using 'pip install tomlkit'.")
"The 'tomlkit' library is not installed. Please install it using 'pip install tomlkit'."
)
sys.stdout.reconfigure(encoding="utf-8", errors="replace") sys.stdout.reconfigure(encoding="utf-8", errors="replace")
@@ -242,15 +239,11 @@ def compare_files(
"ignore" not in sort_ignore_translation[language] "ignore" not in sort_ignore_translation[language]
or len(sort_ignore_translation[language].get("ignore", [])) < 1 or len(sort_ignore_translation[language].get("ignore", [])) < 1
): ):
sort_ignore_translation[language]["ignore"] = tomlkit.array( sort_ignore_translation[language]["ignore"] = tomlkit.array(["language.direction"])
["language.direction"]
)
# Clean up ignore list to only include keys present in reference # Clean up ignore list to only include keys present in reference
sort_ignore_translation[language]["ignore"] = [ sort_ignore_translation[language]["ignore"] = [
key key for key in sort_ignore_translation[language]["ignore"] if key in ref_keys or key == "language.direction"
for key in sort_ignore_translation[language]["ignore"]
if key in ref_keys or key == "language.direction"
] ]
translation_entries = load_translation_entries(file_path) translation_entries = load_translation_entries(file_path)
@@ -264,10 +257,7 @@ def compare_files(
continue continue
file_value = translation_entries[default_key] file_value = translation_entries[default_key]
if ( if default_value == file_value and default_key not in sort_ignore_translation[language]["ignore"]:
default_value == file_value
and default_key not in sort_ignore_translation[language]["ignore"]
):
# Missing translation (same as default and not ignored) # Missing translation (same as default and not ignored)
fails += 1 fails += 1
missing_str_keys.append(default_key) missing_str_keys.append(default_key)
@@ -357,9 +347,7 @@ def main() -> None:
lang_file = lang_input lang_file = lang_input
else: else:
candidate = os.path.join(locales_dir, lang_input) candidate = os.path.join(locales_dir, lang_input)
candidate_with_file = os.path.join( candidate_with_file = os.path.join(locales_dir, lang_input, "translation.toml")
locales_dir, lang_input, "translation.toml"
)
if os.path.exists(candidate): if os.path.exists(candidate):
if os.path.isdir(candidate): if os.path.isdir(candidate):
lang_file = candidate_with_file lang_file = candidate_with_file
scripts/coverage-matrix.py
+9 -17
View File
@@ -46,14 +46,14 @@ import json
import sys import sys
from dataclasses import dataclass, field from dataclasses import dataclass, field
from pathlib import Path from pathlib import Path
from typing import Optional
from defusedxml.ElementTree import ParseError as _XMLParseError
# defusedxml hardens the parser against XXE / billion-laughs / entity- # defusedxml hardens the parser against XXE / billion-laughs / entity-
# expansion attacks. JaCoCo XML on a CI runner is trusted input today, # expansion attacks. JaCoCo XML on a CI runner is trusted input today,
# but using the hardened parser is a one-line change and silences # but using the hardened parser is a one-line change and silences
# scanners that pattern-match on `xml.etree.ElementTree.parse`. # scanners that pattern-match on `xml.etree.ElementTree.parse`.
from defusedxml.ElementTree import parse as _xml_parse from defusedxml.ElementTree import parse as _xml_parse
from defusedxml.ElementTree import ParseError as _XMLParseError
AREAS = ("core", "proprietary", "saas", "desktop") AREAS = ("core", "proprietary", "saas", "desktop")
@@ -69,7 +69,7 @@ class Bucket:
def pct(self) -> float: def pct(self) -> float:
return 100.0 * self.covered / self.total if self.total else 0.0 return 100.0 * self.covered / self.total if self.total else 0.0
def add(self, other: "Bucket") -> None: def add(self, other: Bucket) -> None:
self.covered += other.covered self.covered += other.covered
self.total += other.total self.total += other.total
@@ -78,9 +78,7 @@ class Bucket:
class RowBuckets: class RowBuckets:
"""Per-area buckets for one row of the matrix.""" """Per-area buckets for one row of the matrix."""
by_area: dict[str, Bucket] = field( by_area: dict[str, Bucket] = field(default_factory=lambda: {a: Bucket() for a in AREAS})
default_factory=lambda: {a: Bucket() for a in AREAS}
)
# Some inputs (Playwright V8) don't have source-path info, so they # Some inputs (Playwright V8) don't have source-path info, so they
# only contribute to ALL without an area attribution. Track those # only contribute to ALL without an area attribution. Track those
# separately so per-area cells stay honest. # separately so per-area cells stay honest.
@@ -94,7 +92,7 @@ class RowBuckets:
agg.add(self.unattributed) agg.add(self.unattributed)
return agg return agg
def merge(self, other: "RowBuckets") -> None: def merge(self, other: RowBuckets) -> None:
for area in AREAS: for area in AREAS:
self.by_area[area].add(other.by_area[area]) self.by_area[area].add(other.by_area[area])
self.unattributed.add(other.unattributed) self.unattributed.add(other.unattributed)
@@ -103,7 +101,7 @@ class RowBuckets:
# --------------------------------------------------------------------- jacoco # --------------------------------------------------------------------- jacoco
def _classify_backend(package_name: str) -> Optional[str]: def _classify_backend(package_name: str) -> str | None:
"""Map a JaCoCo package name to an area, or None to skip.""" """Map a JaCoCo package name to an area, or None to skip."""
if not package_name: if not package_name:
return None return None
@@ -152,7 +150,7 @@ def parse_jacoco_methods(path: Path) -> RowBuckets:
# --------------------------------------------------------------- vitest (frontend) # --------------------------------------------------------------- vitest (frontend)
def _classify_frontend(file_path: str) -> Optional[str]: def _classify_frontend(file_path: str) -> str | None:
"""Map a vitest per-file path (anything containing src/<area>/) to an area.""" """Map a vitest per-file path (anything containing src/<area>/) to an area."""
if not file_path: if not file_path:
return None return None
@@ -170,9 +168,7 @@ def parse_vitest_per_file(path: Path) -> RowBuckets:
try: try:
data = json.loads(path.read_text()) data = json.loads(path.read_text())
except (OSError, json.JSONDecodeError) as exc: except (OSError, json.JSONDecodeError) as exc:
print( print(f"::warning::Failed to parse vitest summary {path}: {exc}", file=sys.stderr)
f"::warning::Failed to parse vitest summary {path}: {exc}", file=sys.stderr
)
return row return row
for file_path, metrics in data.items(): for file_path, metrics in data.items():
if file_path == "total": if file_path == "total":
@@ -267,11 +263,7 @@ def main(argv: list[str] | None = None) -> int:
args = parser.parse_args(argv) args = parser.parse_args(argv)
# Build each row from its source(s). # Build each row from its source(s).
fe_e2e = ( fe_e2e = parse_playwright_frontend_total(args.playwright_frontend) if args.playwright_frontend else RowBuckets()
parse_playwright_frontend_total(args.playwright_frontend)
if args.playwright_frontend
else RowBuckets()
)
fe_unit = parse_vitest_per_file(args.vitest) if args.vitest else RowBuckets() fe_unit = parse_vitest_per_file(args.vitest) if args.vitest else RowBuckets()
fe_all = RowBuckets() fe_all = RowBuckets()
fe_all.merge(fe_unit) fe_all.merge(fe_unit)
scripts/coverage-summary.py
+5 -6
View File
@@ -21,16 +21,17 @@ import argparse
import json import json
import os import os
import sys import sys
from collections.abc import Iterable
from dataclasses import dataclass from dataclasses import dataclass
from pathlib import Path from pathlib import Path
from typing import Iterable
from defusedxml.ElementTree import ParseError as _XMLParseError
# `defusedxml` swaps out the stdlib expat parser for one that rejects the # `defusedxml` swaps out the stdlib expat parser for one that rejects the
# usual XML attack vectors (XXE / billion laughs / entity expansion). Even # usual XML attack vectors (XXE / billion laughs / entity expansion). Even
# though JaCoCo XML on a CI runner is trusted input, swapping the parser is # though JaCoCo XML on a CI runner is trusted input, swapping the parser is
# a one-line change that silences security scanners and costs nothing. # a one-line change that silences security scanners and costs nothing.
from defusedxml.ElementTree import parse as _xml_parse from defusedxml.ElementTree import parse as _xml_parse
from defusedxml.ElementTree import ParseError as _XMLParseError
JACOCO_COUNTERS = ("LINE", "BRANCH", "METHOD", "CLASS", "INSTRUCTION", "COMPLEXITY") JACOCO_COUNTERS = ("LINE", "BRANCH", "METHOD", "CLASS", "INSTRUCTION", "COMPLEXITY")
@@ -48,7 +49,7 @@ class CounterTotals:
def pct(self) -> float: def pct(self) -> float:
return 100.0 * self.covered / self.total if self.total else 0.0 return 100.0 * self.covered / self.total if self.total else 0.0
def add(self, other: "CounterTotals") -> None: def add(self, other: CounterTotals) -> None:
self.covered += other.covered self.covered += other.covered
self.missed += other.missed self.missed += other.missed
@@ -108,9 +109,7 @@ def render_jacoco(reports: Iterable[tuple[str, Path]]) -> str:
return body return body
lines: list[str] = [] lines: list[str] = []
lines.append( lines.append("| Metric | " + " | ".join(label for label, _ in rows) + " | **Aggregate** |")
"| Metric | " + " | ".join(label for label, _ in rows) + " | **Aggregate** |"
)
lines.append("|---" * (len(rows) + 2) + "|") lines.append("|---" * (len(rows) + 2) + "|")
for t in ("LINE", "BRANCH", "METHOD", "CLASS"): for t in ("LINE", "BRANCH", "METHOD", "CLASS"):
scripts/download_pdf_samples.py
+8 -16
View File
@@ -27,7 +27,6 @@ import os
import re import re
import sys import sys
from pathlib import Path from pathlib import Path
from typing import List, Optional, Set, Tuple
from urllib.parse import unquote, urlparse from urllib.parse import unquote, urlparse
import requests import requests
@@ -71,9 +70,9 @@ def parse_args() -> argparse.Namespace:
return parser.parse_args() return parser.parse_args()
def load_urls(args: argparse.Namespace) -> List[str]: def load_urls(args: argparse.Namespace) -> list[str]:
urls: List[str] = [] urls: list[str] = []
seen: Set[str] = set() seen: set[str] = set()
def add(url: str) -> None: def add(url: str) -> None:
clean = url.strip() clean = url.strip()
@@ -125,7 +124,7 @@ def download_pdf(
output_dir: Path, output_dir: Path,
timeout: int, timeout: int,
overwrite: bool, overwrite: bool,
) -> Tuple[str, Optional[Path], Optional[str]]: ) -> tuple[str, Path | None, str | None]:
try: try:
dest = build_filename(url, output_dir) dest = build_filename(url, output_dir)
if dest.exists() and not overwrite: if dest.exists() and not overwrite:
@@ -161,20 +160,15 @@ def main() -> None:
output_dir = Path(args.output_dir).resolve() output_dir = Path(args.output_dir).resolve()
output_dir.mkdir(parents=True, exist_ok=True) output_dir.mkdir(parents=True, exist_ok=True)
print( print(f"Downloading {len(urls)} PDFs to {output_dir} using {args.workers} workers...")
f"Downloading {len(urls)} PDFs to {output_dir} using {args.workers} workers..."
)
successes = 0 successes = 0
skipped = 0 skipped = 0
failures: List[Tuple[str, str]] = [] failures: list[tuple[str, str]] = []
with concurrent.futures.ThreadPoolExecutor(max_workers=args.workers) as executor: with concurrent.futures.ThreadPoolExecutor(max_workers=args.workers) as executor:
future_to_url = { future_to_url = {
executor.submit( executor.submit(download_pdf, url, output_dir, args.timeout, args.overwrite): url for url in urls
download_pdf, url, output_dir, args.timeout, args.overwrite
): url
for url in urls
} }
for future in concurrent.futures.as_completed(future_to_url): for future in concurrent.futures.as_completed(future_to_url):
url = future_to_url[future] url = future_to_url[future]
@@ -190,9 +184,7 @@ def main() -> None:
print(f"[OK] {url} -> {path}") print(f"[OK] {url} -> {path}")
print() print()
print( print(f"Completed. Success: {successes}, Skipped: {skipped}, Failures: {len(failures)}")
f"Completed. Success: {successes}, Skipped: {skipped}, Failures: {len(failures)}"
)
if failures: if failures:
print("Failures:") print("Failures:")
for url, error in failures: for url, error in failures:
scripts/generate_requirements.bat
-48
View File
@@ -1,48 +0,0 @@
@echo off
REM --------------------------------------------------
REM Batch script to (re-)generate all requirements
REM with check for pip-compile and user confirmation
REM --------------------------------------------------
REM Check if pip-compile is available
pip-compile --version >nul 2>&1
if %ERRORLEVEL% neq 0 (
echo ERROR: pip-compile was not found.
echo Please install pip-tools:
echo pip install pip-tools
echo and ensure that pip-compile is in your PATH.
pause
exit /b 1
)
echo pip-compile detected.
REM Prompt user for confirmation (default = Yes on ENTER)
set /p confirm="Do you want to generate all requirements? [Y/n] "
if /I "%confirm%"=="" set confirm=Y
if /I not "%confirm%"=="Y" (
echo Generation cancelled by user.
pause
exit /b 0
)
echo Starting generation...
echo Generating .github\scripts\requirements_dev.txt
pip-compile --allow-unsafe --generate-hashes --upgrade --strip-extras ^
--output-file=".github\scripts\requirements_dev.txt" ^
".github\scripts\requirements_dev.in"
echo Generating .github\scripts\requirements_sync_readme.txt
pip-compile --generate-hashes --upgrade --strip-extras ^
--output-file=".github\scripts\requirements_sync_readme.txt" ^
".github\scripts\requirements_sync_readme.in"
echo Generating testing\cucumber\requirements.txt
pip-compile --generate-hashes --upgrade --strip-extras ^
--output-file="testing\cucumber\requirements.txt" ^
"testing\cucumber\requirements.in"
echo All done!
pause
scripts/harvest_type3_fonts.py
+13 -23
View File
@@ -27,16 +27,14 @@ import re
import shlex import shlex
import subprocess import subprocess
import sys import sys
from collections.abc import Sequence
from pathlib import Path from pathlib import Path
from typing import Dict, List, Sequence, Tuple
REPO_ROOT = Path(__file__).resolve().parents[1] REPO_ROOT = Path(__file__).resolve().parents[1]
def parse_args() -> argparse.Namespace: def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser( parser = argparse.ArgumentParser(description="Bulk collect Type3 font signatures from PDFs.")
description="Bulk collect Type3 font signatures from PDFs."
)
parser.add_argument( parser.add_argument(
"--input", "--input",
nargs="+", nargs="+",
@@ -72,8 +70,8 @@ def parse_args() -> argparse.Namespace:
return parser.parse_args() return parser.parse_args()
def discover_pdfs(paths: Sequence[str]) -> List[Path]: def discover_pdfs(paths: Sequence[str]) -> list[Path]:
pdfs: List[Path] = [] pdfs: list[Path] = []
for raw in paths: for raw in paths:
path = Path(raw).resolve() path = Path(raw).resolve()
if path.is_file(): if path.is_file():
@@ -113,8 +111,8 @@ def load_signature_file(path: Path) -> dict:
return json.load(handle) return json.load(handle)
def collect_known_signatures(signatures_dir: Path) -> Dict[str, dict]: def collect_known_signatures(signatures_dir: Path) -> dict[str, dict]:
known: Dict[str, dict] = {} known: dict[str, dict] = {}
if not signatures_dir.exists(): if not signatures_dir.exists():
return known return known
for json_file in signatures_dir.rglob("*.json"): for json_file in signatures_dir.rglob("*.json"):
@@ -139,9 +137,7 @@ def collect_known_signatures(signatures_dir: Path) -> Dict[str, dict]:
return known return known
def run_signature_tool( def run_signature_tool(gradle_cmd: str, pdf: Path, output_path: Path, pretty: bool, cwd: Path) -> None:
gradle_cmd: str, pdf: Path, output_path: Path, pretty: bool, cwd: Path
) -> None:
output_path.parent.mkdir(parents=True, exist_ok=True) output_path.parent.mkdir(parents=True, exist_ok=True)
args = f"--pdf {shlex.quote(str(pdf))} --output {shlex.quote(str(output_path))}" args = f"--pdf {shlex.quote(str(pdf))} --output {shlex.quote(str(output_path))}"
if pretty: if pretty:
@@ -157,12 +153,10 @@ def run_signature_tool(
text=True, text=True,
) )
if completed.returncode != 0: if completed.returncode != 0:
raise RuntimeError( raise RuntimeError(f"Gradle Type3SignatureTool failed for {pdf}:\n{completed.stderr.strip()}")
f"Gradle Type3SignatureTool failed for {pdf}:\n{completed.stderr.strip()}"
)
def extract_fonts_from_payload(payload: dict) -> List[dict]: def extract_fonts_from_payload(payload: dict) -> list[dict]:
pdf = payload.get("pdf") pdf = payload.get("pdf")
fonts = [] fonts = []
for font in payload.get("fonts", []): for font in payload.get("fonts", []):
@@ -182,7 +176,7 @@ def extract_fonts_from_payload(payload: dict) -> List[dict]:
return fonts return fonts
def write_report(report_path: Path, fonts_by_signature: Dict[str, dict]) -> None: def write_report(report_path: Path, fonts_by_signature: dict[str, dict]) -> None:
ordered = sorted(fonts_by_signature.values(), key=lambda entry: entry["signature"]) ordered = sorted(fonts_by_signature.values(), key=lambda entry: entry["signature"])
report = { report = {
"generatedAt": dt.datetime.utcnow().isoformat(timespec="seconds") + "Z", "generatedAt": dt.datetime.utcnow().isoformat(timespec="seconds") + "Z",
@@ -201,7 +195,7 @@ def main() -> None:
pdfs = discover_pdfs(args.input) pdfs = discover_pdfs(args.input)
known = collect_known_signatures(signatures_dir) known = collect_known_signatures(signatures_dir)
newly_added: List[Tuple[str, str]] = [] newly_added: list[tuple[str, str]] = []
for pdf in pdfs: for pdf in pdfs:
signature_path = derive_signature_path(pdf, signatures_dir) signature_path = derive_signature_path(pdf, signatures_dir)
@@ -209,15 +203,11 @@ def main() -> None:
try: try:
payload = load_signature_file(signature_path) payload = load_signature_file(signature_path)
except Exception as exc: except Exception as exc:
print( print(f"[WARN] Failed to parse cached signature {signature_path}: {exc}")
f"[WARN] Failed to parse cached signature {signature_path}: {exc}"
)
payload = None payload = None
else: else:
try: try:
run_signature_tool( run_signature_tool(args.gradle_cmd, pdf, signature_path, args.pretty, REPO_ROOT)
args.gradle_cmd, pdf, signature_path, args.pretty, REPO_ROOT
)
except Exception as exc: except Exception as exc:
print(f"[ERROR] Harvest failed for {pdf}: {exc}", file=sys.stderr) print(f"[ERROR] Harvest failed for {pdf}: {exc}", file=sys.stderr)
continue continue
scripts/playwright-coverage-summary.py
+1 -5
View File
@@ -157,11 +157,7 @@ def main(argv: list[str] | None = None) -> int:
stats = aggregate(args.dump_dir) stats = aggregate(args.dump_dir)
write_vitest_summary(stats, args.out) write_vitest_summary(stats, args.out)
pct = ( pct = 100.0 * stats["functions_covered"] / stats["functions_total"] if stats["functions_total"] else 0.0
100.0 * stats["functions_covered"] / stats["functions_total"]
if stats["functions_total"]
else 0.0
)
print( print(
f"Aggregated {stats['tests']} tests / {stats['scripts']} scripts: " f"Aggregated {stats['tests']} tests / {stats['scripts']} scripts: "
f"{stats['functions_covered']}/{stats['functions_total']} functions " f"{stats['functions_covered']}/{stats['functions_total']} functions "
scripts/pre-commit/install_gitleaks.py
+5 -13
View File
@@ -2,7 +2,7 @@
"""Download the pinned gitleaks binary into .task/bin, verifying its checksum. """Download the pinned gitleaks binary into .task/bin, verifying its checksum.
gitleaks is a Go binary with no PyPI package, so it can't be locked like the gitleaks is a Go binary with no PyPI package, so it can't be locked like the
other tools (ruff/codespell/toml-sort live in scripts/pre-commit/pyproject.toml). other tools (ruff/codespell/toml-sort live in engine/pyproject.toml).
This script is the single source of truth for the gitleaks version and the This script is the single source of truth for the gitleaks version and the
SHA-256 of each release asset. It is cross-platform (stdlib only) and idempotent: SHA-256 of each release asset. It is cross-platform (stdlib only) and idempotent:
if the cached binary already reports the pinned version it does nothing, so if the cached binary already reports the pinned version it does nothing, so
@@ -40,9 +40,7 @@ BIN = REPO_ROOT / ".task" / "bin" / ("gitleaks.exe" if IS_WINDOWS else "gitleaks
def platform_key() -> str: def platform_key() -> str:
os_name = {"Linux": "linux", "Darwin": "darwin", "Windows": "windows"}.get( os_name = {"Linux": "linux", "Darwin": "darwin", "Windows": "windows"}.get(platform.system())
platform.system()
)
arch = { arch = {
"x86_64": "x64", "x86_64": "x64",
"amd64": "x64", "amd64": "x64",
@@ -55,9 +53,7 @@ def platform_key() -> str:
"armv6l": "armv6", "armv6l": "armv6",
}.get(platform.machine().lower()) }.get(platform.machine().lower())
if not os_name or not arch: if not os_name or not arch:
raise SystemExit( raise SystemExit(f"Unsupported platform for gitleaks: {platform.system()}/{platform.machine()}")
f"Unsupported platform for gitleaks: {platform.system()}/{platform.machine()}"
)
return f"{os_name}_{arch}" return f"{os_name}_{arch}"
@@ -65,9 +61,7 @@ def cached_version() -> str | None:
if not BIN.exists(): if not BIN.exists():
return None return None
try: try:
return subprocess.run( return subprocess.run([str(BIN), "version"], capture_output=True, text=True).stdout.strip()
[str(BIN), "version"], capture_output=True, text=True
).stdout.strip()
except OSError: except OSError:
return None return None
@@ -90,9 +84,7 @@ def main() -> int:
archive, _ = urllib.request.urlretrieve(url) archive, _ = urllib.request.urlretrieve(url)
digest = hashlib.sha256(Path(archive).read_bytes()).hexdigest() digest = hashlib.sha256(Path(archive).read_bytes()).hexdigest()
if digest != expected: if digest != expected:
raise SystemExit( raise SystemExit(f"gitleaks checksum mismatch: expected {expected}, got {digest}")
f"gitleaks checksum mismatch: expected {expected}, got {digest}"
)
member = "gitleaks.exe" if IS_WINDOWS else "gitleaks" member = "gitleaks.exe" if IS_WINDOWS else "gitleaks"
if suffix == "zip": if suffix == "zip":
scripts/pre-commit/pyproject.toml
-16
View File
@@ -1,16 +0,0 @@
# Pinned Python lint/format tools for `task pre-commit`. uv.lock locks these
# plus their transitive dependencies by hash, so `uv run --project
# scripts/pre-commit --locked <tool>` is reproducible and integrity-checked.
# This is not a packaged project - it only exists to lock the tooling.
[project]
name = "stirling-precommit-tools"
version = "0"
requires-python = ">=3.11"
dependencies = [
"ruff==0.15.14",
"codespell==2.4.2",
"tomli-w==1.2.0",
]
[tool.uv]
package = false
scripts/pre-commit/sort_locale_toml.py
+2 -6
View File
@@ -54,13 +54,9 @@ def sort_file(path: str, fix: bool) -> bool:
try: try:
reordered = tomllib.loads(expected) reordered = tomllib.loads(expected)
except tomllib.TOMLDecodeError as exc: except tomllib.TOMLDecodeError as exc:
raise SortError( raise SortError(f"{path}: refusing to sort, the sorted output is not valid TOML: {exc}") from exc
f"{path}: refusing to sort, the sorted output is not valid TOML: {exc}"
) from exc
if reordered != original: if reordered != original:
raise SortError( raise SortError(f"{path}: refusing to sort, sorting would change the file's contents")
f"{path}: refusing to sort, sorting would change the file's contents"
)
if fix: if fix:
Path(path).write_text(expected, encoding="utf-8") Path(path).write_text(expected, encoding="utf-8")
scripts/pre-commit/uv.lock
Generated
-63
View File
@@ -1,63 +0,0 @@
version = 1
revision = 3
requires-python = ">=3.11"
[[package]]
name = "codespell"
version = "2.4.2"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/2d/9d/1d0903dff693160f893ca6abcabad545088e7a2ee0a6deae7c24e958be69/codespell-2.4.2.tar.gz", hash = "sha256:3c33be9ae34543807f088aeb4832dfad8cb2dae38da61cac0a7045dd376cfdf3", size = 352058, upload-time = "2026-03-05T18:10:42.936Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/42/a1/52fa05533e95fe45bcc09bcf8a503874b1c08f221a4e35608017e0938f55/codespell-2.4.2-py3-none-any.whl", hash = "sha256:97e0c1060cf46bd1d5db89a936c98db8c2b804e1fdd4b5c645e82a1ec6b1f886", size = 353715, upload-time = "2026-03-05T18:10:41.398Z" },
]
[[package]]
name = "ruff"
version = "0.15.14"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/dc/8a/8bce2894573e9dae6ff4d77fe34ad727d79b9e6238ad288c5638990d90f6/ruff-0.15.14.tar.gz", hash = "sha256:48e866b165be4a9bdbf310f7d3c9a07edef2fe8cd63ffeb4e00bb590506ebf9f", size = 4700910, upload-time = "2026-05-21T14:34:55.177Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/b9/c8/74a92c6ff9fcfb4f1f947126d3ebee8389276e161ecc85de5bda7cda51bd/ruff-0.15.14-py3-none-linux_armv6l.whl", hash = "sha256:8dd2db9416e487c8d4b01fa7056bb02c4d05969d4f8d17a08c229c2f4ff3c108", size = 10739177, upload-time = "2026-05-21T14:34:37.332Z" },
{ url = "https://files.pythonhosted.org/packages/45/91/254a35c20acc38a7223c9d2d594af12e794432464f2cdeb52af1dc4a892d/ruff-0.15.14-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:be4ff55af755bd71a00ab3dc6bd7ffc467bd76e0df6881e286c2e3d23e8fb43b", size = 11144969, upload-time = "2026-05-21T14:34:43.978Z" },
{ url = "https://files.pythonhosted.org/packages/56/9e/d13e40f83b8d0a94430e6778ce1d94a43b38cf2efe63278bdd2b4c65abbf/ruff-0.15.14-py3-none-macosx_11_0_arm64.whl", hash = "sha256:48d5909d7d06276ce7dde6d32bfa4b0d4cb2651145cd8ee4b440722cbc77832f", size = 10478207, upload-time = "2026-05-21T14:34:48.378Z" },
{ url = "https://files.pythonhosted.org/packages/8d/f1/b15a7839fa4f332f8acec78e20564f26bb2d866e3d21710b877fd0263000/ruff-0.15.14-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:ca8cbfa94c4f90984a67561978602746d4cd27103568f745fa90eee3f0d4107d", size = 10818459, upload-time = "2026-05-21T14:34:22.318Z" },
{ url = "https://files.pythonhosted.org/packages/45/33/53d651177f84f94b400a0e27f8824eeada3dddc9d5ee8aeb048f4352a520/ruff-0.15.14-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:9a6bbc0333f1ab053423bcbf6226477d266ca7cec7738c4c8e3f55647803f3c4", size = 10541800, upload-time = "2026-05-21T14:34:20.209Z" },
{ url = "https://files.pythonhosted.org/packages/b8/a6/868f87e0bf9786ed24b5d0d0ad8676b8a94fd1912f42cddf9cfc7857818a/ruff-0.15.14-py3-none-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:8a24a4f7605d7003a6674d4387651effd939dead3fddd0f36561eb77a9a2e542", size = 11342149, upload-time = "2026-05-21T14:34:46.365Z" },
{ url = "https://files.pythonhosted.org/packages/a7/8b/38cd5c19faffdcc05a408d2b78edccc69492ab9720eadb49ea15ef80d768/ruff-0.15.14-py3-none-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:049b5326e53ed80978f2fc041a280603f69dd6b0c95464342a2bb4572d9d9e2f", size = 12212563, upload-time = "2026-05-21T14:34:28.579Z" },
{ url = "https://files.pythonhosted.org/packages/3e/4d/a3c5b874a556d5731e3e657aaf04311bb76f0a5c3ec220ed43051be6b64b/ruff-0.15.14-py3-none-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:d4ed42e6696c8dfa5f06728e6441993901f548eb92d73bc472cb5a38d1395fbf", size = 11493299, upload-time = "2026-05-21T14:34:41.836Z" },
{ url = "https://files.pythonhosted.org/packages/1e/c0/56472c251d09858a53e51efbd485b09e1995d8731668b76d52e5dd6ee0f1/ruff-0.15.14-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:715c543cf450c4888251f91c52f1942a800541d9bddd7ac060aa4e6b77ae7cba", size = 11455931, upload-time = "2026-05-21T14:34:57.276Z" },
{ url = "https://files.pythonhosted.org/packages/2c/4a/e2e7b4d8dbf233d4eace59c75bc3435fa6d8bd3bae82d351d4e4300c0fd1/ruff-0.15.14-py3-none-manylinux_2_31_riscv64.whl", hash = "sha256:72ebab6013ec887d439d8b7593737a0a4ffb06d45d209d4e4bf2e92813082d3f", size = 11400794, upload-time = "2026-05-21T14:34:39.773Z" },
{ url = "https://files.pythonhosted.org/packages/97/c7/83c0539fe34c3e09136204d1e75d6052492364e0b3cb05e9465423f567d7/ruff-0.15.14-py3-none-musllinux_1_2_aarch64.whl", hash = "sha256:49072d36abdbe97a8dd7f480afe9c675699c0c495d4c84076e2c1203c4550581", size = 10804759, upload-time = "2026-05-21T14:34:31.045Z" },
{ url = "https://files.pythonhosted.org/packages/86/a6/18f2bfc095a2ab4a78745644e428205532ce6653a5d0fa8501572891534d/ruff-0.15.14-py3-none-musllinux_1_2_armv7l.whl", hash = "sha256:958522aee105068640c2c2ceae08f413ae44d922f52a1374ac13d6a96032fc93", size = 10539517, upload-time = "2026-05-21T14:34:53.064Z" },
{ url = "https://files.pythonhosted.org/packages/54/3a/5a8b3b69c654d4e4bf1d246ac5b49cbcdac6eaab6905925f8915f31e3b80/ruff-0.15.14-py3-none-musllinux_1_2_i686.whl", hash = "sha256:f3707da619a143a2e8830e2abab8224478d69ace2d28cb6c20543ae97c36bf61", size = 11065169, upload-time = "2026-05-21T14:34:24.484Z" },
{ url = "https://files.pythonhosted.org/packages/ed/c5/8864e4e7925b836ea354b31d57641ec03830564e281a8b6f061f8c3e0ec1/ruff-0.15.14-py3-none-musllinux_1_2_x86_64.whl", hash = "sha256:bb01d645694e3ec0102105d07ef2d53703970407d59c04e59d3ba0b7a1d53553", size = 11560214, upload-time = "2026-05-21T14:34:50.975Z" },
{ url = "https://files.pythonhosted.org/packages/36/38/012bf76752e1f89ed50b77b99532d90f3a3e287bc7918e1fc0948ac866ac/ruff-0.15.14-py3-none-win32.whl", hash = "sha256:6d0c1ad2a0ab718d39b6d8fd2217981ce4d625cd96a720095f798fb47d8b13e6", size = 10805548, upload-time = "2026-05-21T14:34:33.453Z" },
{ url = "https://files.pythonhosted.org/packages/d1/b7/4ea2c170f10ad760fff2a5250beb18897719dc8b52b53a24cddbb9dd3f19/ruff-0.15.14-py3-none-win_amd64.whl", hash = "sha256:802342981e056db3851a7836e5b070f8f15f67d4a685ae2a6160939d364b2902", size = 11939523, upload-time = "2026-05-21T14:34:18.077Z" },
{ url = "https://files.pythonhosted.org/packages/62/d5/bc97ff895ec35cf3925d4bd60f3b39d822f377a446906ec9bcc87405e59b/ruff-0.15.14-py3-none-win_arm64.whl", hash = "sha256:ff47b90a9ef6a40c9e2f3b479c1fb78531adf055b94c1eba0a7ba04b31951826", size = 11208607, upload-time = "2026-05-21T14:34:26.525Z" },
]
[[package]]
name = "stirling-precommit-tools"
version = "0"
source = { virtual = "." }
dependencies = [
{ name = "codespell" },
{ name = "ruff" },
{ name = "tomli-w" },
]
[package.metadata]
requires-dist = [
{ name = "codespell", specifier = "==2.4.2" },
{ name = "ruff", specifier = "==0.15.14" },
{ name = "tomli-w", specifier = "==1.2.0" },
]
[[package]]
name = "tomli-w"
version = "1.2.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/19/75/241269d1da26b624c0d5e110e8149093c759b7a286138f4efd61a60e75fe/tomli_w-1.2.0.tar.gz", hash = "sha256:2dd14fac5a47c27be9cd4c976af5a12d87fb1f0b4512f81d69cce3b35ae25021", size = 7184, upload-time = "2025-01-15T12:07:24.262Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/c7/18/c86eb8e0202e32dd3df50d43d7ff9854f8e0603945ff398974c1d91ac1ef/tomli_w-1.2.0-py3-none-any.whl", hash = "sha256:188306098d013b691fcadc011abd66727d3c414c571bb01b1a174ba8c983cf90", size = 6675, upload-time = "2025-01-15T12:07:22.074Z" },
]
scripts/pre-commit/whitespace.py
+4 -1
View File
@@ -40,7 +40,10 @@ def normalise(data: bytes) -> bytes:
body = b"\n".join(lines) body = b"\n".join(lines)
# Ensure a non-empty file ends with exactly one newline. # Ensure a non-empty file ends with exactly one newline.
stripped = body.rstrip(b"\r\n") stripped = body.rstrip(b"\r\n")
return stripped + b"\n" if stripped else body if not stripped:
return body
newline = b"\r\n" if data.endswith(b"\r\n") else b"\n"
return stripped + newline
def main() -> int: def main() -> int:
scripts/summarize_type3_signatures.py
+7 -15
View File
@@ -14,13 +14,10 @@ import argparse
import json import json
from collections import defaultdict from collections import defaultdict
from pathlib import Path from pathlib import Path
from typing import Dict, List
def parse_args() -> argparse.Namespace: def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser( parser = argparse.ArgumentParser(description="Summarize Type3 signature JSON dumps.")
description="Summarize Type3 signature JSON dumps."
)
parser.add_argument( parser.add_argument(
"--input", "--input",
default="docs/type3/signatures", default="docs/type3/signatures",
@@ -34,8 +31,8 @@ def parse_args() -> argparse.Namespace:
return parser.parse_args() return parser.parse_args()
def load_signatures(directory: Path) -> Dict[str, List[dict]]: def load_signatures(directory: Path) -> dict[str, list[dict]]:
inventory: Dict[str, List[dict]] = defaultdict(list) inventory: dict[str, list[dict]] = defaultdict(list)
for path in sorted(directory.glob("*.json")): for path in sorted(directory.glob("*.json")):
with path.open("r", encoding="utf-8") as handle: with path.open("r", encoding="utf-8") as handle:
payload = json.load(handle) payload = json.load(handle)
@@ -55,15 +52,12 @@ def load_signatures(directory: Path) -> Dict[str, List[dict]]:
return inventory return inventory
def write_markdown( def write_markdown(inventory: dict[str, list[dict]], output: Path, input_dir: Path) -> None:
inventory: Dict[str, List[dict]], output: Path, input_dir: Path lines: list[str] = []
) -> None:
lines: List[str] = []
lines.append("# Type3 Signature Inventory") lines.append("# Type3 Signature Inventory")
lines.append("") lines.append("")
lines.append( lines.append(
f"_Generated from `{input_dir}`. " f"_Generated from `{input_dir}`. Run `scripts/summarize_type3_signatures.py` after capturing new samples._"
"Run `scripts/summarize_type3_signatures.py` after capturing new samples._"
) )
lines.append("") lines.append("")
@@ -76,9 +70,7 @@ def write_markdown(
for entry in entries: for entry in entries:
signature = entry.get("signature") or "—" signature = entry.get("signature") or "—"
sample = Path(entry["source"]).name sample = Path(entry["source"]).name
glyph_count = ( glyph_count = entry.get("glyphCount") if entry.get("glyphCount") is not None else "—"
entry.get("glyphCount") if entry.get("glyphCount") is not None else "—"
)
coverage = entry.get("glyphCoverage") or [] coverage = entry.get("glyphCoverage") or []
preview = ", ".join(str(code) for code in coverage[:10]) preview = ", ".join(str(code) for code in coverage[:10])
lines.append(f"| `{signature}` | `{sample}` | {glyph_count} | {preview} |") lines.append(f"| `{signature}` | `{sample}` | {glyph_count} | {preview} |")
scripts/sync_en_us_spelling.py
+4 -12
View File
@@ -382,9 +382,7 @@ def make_converter(mapping: dict[str, str]):
return lambda text: (text, []) return lambda text: (text, [])
# Longest-first so multi-word/longer forms win; \b ensures whole words. # Longest-first so multi-word/longer forms win; \b ensures whole words.
pattern = re.compile( pattern = re.compile(
r"\b(" r"\b(" + "|".join(re.escape(w) for w in sorted(mapping, key=len, reverse=True)) + r")\b",
+ "|".join(re.escape(w) for w in sorted(mapping, key=len, reverse=True))
+ r")\b",
re.IGNORECASE, re.IGNORECASE,
) )
@@ -509,9 +507,7 @@ def parse_structured(
continue continue
kv = KV_RE.match(s) kv = KV_RE.match(s)
if kv: if kv:
(top if section == "" else sections[section]).append( (top if section == "" else sections[section]).append((kv.group(1), kv.group(2)))
(kv.group(1), kv.group(2))
)
return top, order, sections return top, order, sections
@@ -563,9 +559,7 @@ def sync_en_us(dry_run: bool) -> int:
# en-US-only keys that belong to this (shared) section # en-US-only keys that belong to this (shared) section
for k, v in us_sections.get(name, []): for k, v in us_sections.get(name, []):
if k not in gb_section_keys: if k not in gb_section_keys:
_insert_ci( _insert_ci(merged, (k, uk_to_us_convert(v)[0]), lambda kv: kv[0].lower())
merged, (k, uk_to_us_convert(v)[0]), lambda kv: kv[0].lower()
)
out_sections.append((name, merged)) out_sections.append((name, merged))
# en-US-only sections (absent from en-GB): insert by ci header order # en-US-only sections (absent from en-GB): insert by ci header order
@@ -595,9 +589,7 @@ def sync_en_us(dry_run: bool) -> int:
def main() -> int: def main() -> int:
ap = argparse.ArgumentParser(description=__doc__) ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument( ap.add_argument("--dry-run", action="store_true", help="report changes without writing")
"--dry-run", action="store_true", help="report changes without writing"
)
args = ap.parse_args() args = ap.parse_args()
if not EN_US.exists() or not EN_GB.exists(): if not EN_US.exists() or not EN_GB.exists():
scripts/translations/ai_translation_helper.py
+39 -92
View File
@@ -6,14 +6,15 @@ batch processing, quality checks, and integration helpers.
TOML format only. TOML format only.
""" """
import json
from pathlib import Path
from typing import Dict, List, Any
import argparse import argparse
import re
from datetime import datetime
import csv import csv
import json
import re
import tomllib import tomllib
from datetime import datetime
from pathlib import Path
from typing import Any
import tomli_w import tomli_w
@@ -22,7 +23,7 @@ class AITranslationHelper:
self.locales_dir = Path(locales_dir) self.locales_dir = Path(locales_dir)
self.golden_truth_file = self.locales_dir / "en-US" / "translation.toml" self.golden_truth_file = self.locales_dir / "en-US" / "translation.toml"
def _load_translation_file(self, file_path: Path) -> Dict: def _load_translation_file(self, file_path: Path) -> dict:
"""Load TOML translation file.""" """Load TOML translation file."""
try: try:
with open(file_path, "rb") as f: with open(file_path, "rb") as f:
@@ -31,14 +32,14 @@ class AITranslationHelper:
print(f"Error loading {file_path}: {e}") print(f"Error loading {file_path}: {e}")
return {} return {}
def _save_translation_file(self, data: Dict, file_path: Path) -> None: def _save_translation_file(self, data: dict, file_path: Path) -> None:
"""Save TOML translation file.""" """Save TOML translation file."""
with open(file_path, "wb") as f: with open(file_path, "wb") as f:
tomli_w.dump(data, f) tomli_w.dump(data, f)
def create_ai_batch_file( def create_ai_batch_file(
self, self,
languages: List[str], languages: list[str],
output_file: Path, output_file: Path,
max_entries_per_language: int = 50, max_entries_per_language: int = 50,
) -> None: ) -> None:
@@ -74,14 +75,9 @@ class AITranslationHelper:
untranslated = self._find_untranslated_entries(golden_truth, lang_data) untranslated = self._find_untranslated_entries(golden_truth, lang_data)
# Limit entries if specified # Limit entries if specified
if ( if max_entries_per_language and len(untranslated) > max_entries_per_language:
max_entries_per_language
and len(untranslated) > max_entries_per_language
):
# Prioritize by key importance # Prioritize by key importance
untranslated = self._prioritize_translation_keys( untranslated = self._prioritize_translation_keys(untranslated, max_entries_per_language)
untranslated, max_entries_per_language
)
batch_data["translations"][lang] = {} batch_data["translations"][lang] = {}
for key, value in untranslated.items(): for key, value in untranslated.items():
@@ -94,15 +90,11 @@ class AITranslationHelper:
# Always save batch files as JSON for compatibility # Always save batch files as JSON for compatibility
with open(output_file, "w", encoding="utf-8") as f: with open(output_file, "w", encoding="utf-8") as f:
json.dump(batch_data, f, indent=2, ensure_ascii=False) json.dump(batch_data, f, indent=2, ensure_ascii=False)
total_entries = sum( total_entries = sum(len(lang_data) for lang_data in batch_data["translations"].values())
len(lang_data) for lang_data in batch_data["translations"].values()
)
print(f"Created AI batch file: {output_file}") print(f"Created AI batch file: {output_file}")
print(f"Total entries to translate: {total_entries}") print(f"Total entries to translate: {total_entries}")
def _find_untranslated_entries( def _find_untranslated_entries(self, golden_truth: dict, lang_data: dict) -> dict[str, str]:
self, golden_truth: Dict, lang_data: Dict
) -> Dict[str, str]:
"""Find entries that need translation.""" """Find entries that need translation."""
golden_flat = self._flatten_dict(golden_truth) golden_flat = self._flatten_dict(golden_truth)
lang_flat = self._flatten_dict(lang_data) lang_flat = self._flatten_dict(lang_data)
@@ -112,19 +104,14 @@ class AITranslationHelper:
if ( if (
key not in lang_flat key not in lang_flat
or lang_flat[key] == value or lang_flat[key] == value
or ( or (isinstance(lang_flat[key], str) and lang_flat[key].startswith("[UNTRANSLATED]"))
isinstance(lang_flat[key], str)
and lang_flat[key].startswith("[UNTRANSLATED]")
)
): ):
if not self._is_expected_identical(key, value): if not self._is_expected_identical(key, value):
untranslated[key] = value untranslated[key] = value
return untranslated return untranslated
def _flatten_dict( def _flatten_dict(self, d: dict, parent_key: str = "", separator: str = ".") -> dict[str, Any]:
self, d: Dict, parent_key: str = "", separator: str = "."
) -> Dict[str, Any]:
"""Flatten nested dictionary.""" """Flatten nested dictionary."""
items = [] items = []
for k, v in d.items(): for k, v in d.items():
@@ -141,9 +128,7 @@ class AITranslationHelper:
return True return True
return "language.direction" in key.lower() return "language.direction" in key.lower()
def _prioritize_translation_keys( def _prioritize_translation_keys(self, untranslated: dict[str, str], max_count: int) -> dict[str, str]:
self, untranslated: Dict[str, str], max_count: int
) -> Dict[str, str]:
"""Prioritize which keys to translate first based on importance.""" """Prioritize which keys to translate first based on importance."""
# Define priority order (higher score = higher priority) # Define priority order (higher score = higher priority)
priority_patterns = [ priority_patterns = [
@@ -191,19 +176,17 @@ class AITranslationHelper:
if len(parts) > 0: if len(parts) > 0:
main_section = parts[0] main_section = parts[0]
context = contexts.get( context = contexts.get(main_section, f"Part of {main_section} functionality")
main_section, f"Part of {main_section} functionality"
)
if len(parts) > 1: if len(parts) > 1:
context += f", specifically for {parts[-1]}" context += f", specifically for {parts[-1]}"
return context return context
return "General application text" return "General application text"
def validate_ai_translations(self, batch_file: Path) -> Dict[str, List[str]]: def validate_ai_translations(self, batch_file: Path) -> dict[str, list[str]]:
"""Validate AI translations for common issues.""" """Validate AI translations for common issues."""
# Batch files are always JSON # Batch files are always JSON
with open(batch_file, "r", encoding="utf-8") as f: with open(batch_file, encoding="utf-8") as f:
batch_data = json.load(f) batch_data = json.load(f)
issues = {"errors": [], "warnings": []} issues = {"errors": [], "warnings": []}
@@ -227,29 +210,21 @@ class AITranslationHelper:
) )
# Check if translation is identical to original (might be untranslated) # Check if translation is identical to original (might be untranslated)
if translated == original and not self._is_expected_identical( if translated == original and not self._is_expected_identical(key, original):
key, original issues["warnings"].append(f"{lang}.{key}: Translation identical to original")
):
issues["warnings"].append(
f"{lang}.{key}: Translation identical to original"
)
# Check for common AI translation artifacts # Check for common AI translation artifacts
artifacts = ["[TRANSLATE]", "[TODO]", "UNTRANSLATED", "{{", "}}"] artifacts = ["[TRANSLATE]", "[TODO]", "UNTRANSLATED", "{{", "}}"]
for artifact in artifacts: for artifact in artifacts:
if artifact in translated: if artifact in translated:
issues["errors"].append( issues["errors"].append(f"{lang}.{key}: Contains translation artifact: {artifact}")
f"{lang}.{key}: Contains translation artifact: {artifact}"
)
return issues return issues
def apply_ai_batch_translations( def apply_ai_batch_translations(self, batch_file: Path, validate: bool = True) -> dict[str, Any]:
self, batch_file: Path, validate: bool = True
) -> Dict[str, Any]:
"""Apply translations from AI batch file to individual language files.""" """Apply translations from AI batch file to individual language files."""
# Batch files are always JSON # Batch files are always JSON
with open(batch_file, "r", encoding="utf-8") as f: with open(batch_file, encoding="utf-8") as f:
batch_data = json.load(f) batch_data = json.load(f)
results = {"applied": {}, "errors": [], "warnings": []} results = {"applied": {}, "errors": [], "warnings": []}
@@ -291,7 +266,7 @@ class AITranslationHelper:
return results return results
def _set_nested_value(self, data: Dict, key_path: str, value: Any) -> None: def _set_nested_value(self, data: dict, key_path: str, value: Any) -> None:
"""Set value in nested dict using dot notation.""" """Set value in nested dict using dot notation."""
keys = key_path.split(".") keys = key_path.split(".")
current = data current = data
@@ -300,24 +275,18 @@ class AITranslationHelper:
current[key] = {} current[key] = {}
elif not isinstance(current[key], dict): elif not isinstance(current[key], dict):
# If the current value is not a dict, we can't nest into it # If the current value is not a dict, we can't nest into it
print( print(f"Warning: Converting non-dict value at '{key}' to dict to allow nesting")
f"Warning: Converting non-dict value at '{key}' to dict to allow nesting"
)
current[key] = {} current[key] = {}
current = current[key] current = current[key]
current[keys[-1]] = value current[keys[-1]] = value
def export_for_external_translation( def export_for_external_translation(self, languages: list[str], output_format: str = "csv") -> None:
self, languages: List[str], output_format: str = "csv"
) -> None:
"""Export translations for external translation services.""" """Export translations for external translation services."""
golden_truth = self._load_translation_file(self.golden_truth_file) golden_truth = self._load_translation_file(self.golden_truth_file)
golden_flat = self._flatten_dict(golden_truth) golden_flat = self._flatten_dict(golden_truth)
if output_format == "csv": if output_format == "csv":
output_file = Path( output_file = Path(f"translations_export_{datetime.now().strftime('%Y%m%d')}.csv")
f"translations_export_{datetime.now().strftime('%Y%m%d')}.csv"
)
with open(output_file, "w", newline="", encoding="utf-8") as csvfile: with open(output_file, "w", newline="", encoding="utf-8") as csvfile:
fieldnames = ["key", "context", "en_US"] + languages fieldnames = ["key", "context", "en_US"] + languages
@@ -353,9 +322,7 @@ class AITranslationHelper:
print(f"Exported to {output_file}") print(f"Exported to {output_file}")
elif output_format == "json": elif output_format == "json":
output_file = Path( output_file = Path(f"translations_export_{datetime.now().strftime('%Y%m%d')}.json")
f"translations_export_{datetime.now().strftime('%Y%m%d')}.json"
)
export_data = {"languages": languages, "translations": {}} export_data = {"languages": languages, "translations": {}}
for key, en_value in golden_flat.items(): for key, en_value in golden_flat.items():
@@ -386,9 +353,7 @@ class AITranslationHelper:
def main(): def main():
parser = argparse.ArgumentParser( parser = argparse.ArgumentParser(description="AI Translation Helper", epilog="Works with TOML translation files.")
description="AI Translation Helper", epilog="Works with TOML translation files."
)
parser.add_argument( parser.add_argument(
"--locales-dir", "--locales-dir",
default="frontend/editor/public/locales", default="frontend/editor/public/locales",
@@ -398,40 +363,24 @@ def main():
subparsers = parser.add_subparsers(dest="command", help="Available commands") subparsers = parser.add_subparsers(dest="command", help="Available commands")
# Create batch command # Create batch command
batch_parser = subparsers.add_parser( batch_parser = subparsers.add_parser("create-batch", help="Create AI translation batch file")
"create-batch", help="Create AI translation batch file" batch_parser.add_argument("--languages", nargs="+", required=True, help="Language codes to include")
)
batch_parser.add_argument(
"--languages", nargs="+", required=True, help="Language codes to include"
)
batch_parser.add_argument("--output", required=True, help="Output batch file") batch_parser.add_argument("--output", required=True, help="Output batch file")
batch_parser.add_argument( batch_parser.add_argument("--max-entries", type=int, default=100, help="Max entries per language")
"--max-entries", type=int, default=100, help="Max entries per language"
)
# Validate command # Validate command
validate_parser = subparsers.add_parser("validate", help="Validate AI translations") validate_parser = subparsers.add_parser("validate", help="Validate AI translations")
validate_parser.add_argument("batch_file", help="Batch file to validate") validate_parser.add_argument("batch_file", help="Batch file to validate")
# Apply command # Apply command
apply_parser = subparsers.add_parser( apply_parser = subparsers.add_parser("apply-batch", help="Apply AI batch translations")
"apply-batch", help="Apply AI batch translations"
)
apply_parser.add_argument("batch_file", help="Batch file with translations") apply_parser.add_argument("batch_file", help="Batch file with translations")
apply_parser.add_argument( apply_parser.add_argument("--skip-validation", action="store_true", help="Skip validation before applying")
"--skip-validation", action="store_true", help="Skip validation before applying"
)
# Export command # Export command
export_parser = subparsers.add_parser( export_parser = subparsers.add_parser("export", help="Export for external translation")
"export", help="Export for external translation" export_parser.add_argument("--languages", nargs="+", required=True, help="Language codes to export")
) export_parser.add_argument("--format", choices=["csv", "json"], default="csv", help="Export format")
export_parser.add_argument(
"--languages", nargs="+", required=True, help="Language codes to export"
)
export_parser.add_argument(
"--format", choices=["csv", "json"], default="csv", help="Export format"
)
args = parser.parse_args() args = parser.parse_args()
@@ -464,9 +413,7 @@ def main():
elif args.command == "apply-batch": elif args.command == "apply-batch":
batch_file = Path(args.batch_file) batch_file = Path(args.batch_file)
results = helper.apply_ai_batch_translations( results = helper.apply_ai_batch_translations(batch_file, validate=not args.skip_validation)
batch_file, validate=not args.skip_validation
)
total_applied = sum(results["applied"].values()) total_applied = sum(results["applied"].values())
print(f"Total translations applied: {total_applied}") print(f"Total translations applied: {total_applied}")
scripts/translations/auto_translate.py
+17 -45
View File
@@ -5,16 +5,15 @@ Extracts, translates, merges, and beautifies translations for a language.
TOML format only. TOML format only.
""" """
import json
import sys
import argparse import argparse
import json
import os import os
import subprocess import subprocess
import sys
from concurrent.futures import ThreadPoolExecutor from concurrent.futures import ThreadPoolExecutor
from pathlib import Path
import time import time
import tomllib import tomllib
from pathlib import Path
def run_command(cmd, description=""): def run_command(cmd, description=""):
@@ -50,27 +49,19 @@ def load_translation_file(file_path):
def extract_untranslated(language_code, batch_size=500, include_existing=False): def extract_untranslated(language_code, batch_size=500, include_existing=False):
"""Extract untranslated entries and split into batches.""" """Extract untranslated entries and split into batches."""
mode = ( mode = "all untranslated (including existing)" if include_existing else "new (missing)"
"all untranslated (including existing)" if include_existing else "new (missing)"
)
print(f"\n🔍 Extracting {mode} entries for {language_code}...") print(f"\n🔍 Extracting {mode} entries for {language_code}...")
# Load files # Load files
golden_path = find_translation_file(Path("frontend/editor/public/locales/en-US")) golden_path = find_translation_file(Path("frontend/editor/public/locales/en-US"))
lang_path = find_translation_file( lang_path = find_translation_file(Path(f"frontend/editor/public/locales/{language_code}"))
Path(f"frontend/editor/public/locales/{language_code}")
)
if not golden_path: if not golden_path:
print( print("Error: Golden truth file not found in frontend/editor/public/locales/en-US")
"Error: Golden truth file not found in frontend/editor/public/locales/en-US"
)
return None return None
if not lang_path: if not lang_path:
print( print(f"Error: Language file not found in frontend/editor/public/locales/{language_code}")
f"Error: Language file not found in frontend/editor/public/locales/{language_code}"
)
return None return None
def flatten_dict(d, parent_key="", separator="."): def flatten_dict(d, parent_key="", separator="."):
@@ -101,10 +92,7 @@ def extract_untranslated(language_code, batch_size=500, include_existing=False):
if ( if (
key not in lang_flat key not in lang_flat
or lang_flat.get(key) == value or lang_flat.get(key) == value
or ( or (isinstance(lang_flat.get(key), str) and lang_flat.get(key).startswith("[UNTRANSLATED]"))
isinstance(lang_flat.get(key), str)
and lang_flat.get(key).startswith("[UNTRANSLATED]")
)
): ):
untranslated[key] = value untranslated[key] = value
else: else:
@@ -141,9 +129,7 @@ def extract_untranslated(language_code, batch_size=500, include_existing=False):
return batch_files return batch_files
def translate_batches( def translate_batches(batch_files, language_code, api_key, timeout=600, model="gpt-5.5", parallel=1):
batch_files, language_code, api_key, timeout=600, model="gpt-5.5", parallel=1
):
"""Translate all batch files using the given OpenAI model.""" """Translate all batch files using the given OpenAI model."""
if not batch_files: if not batch_files:
return [] return []
@@ -169,9 +155,7 @@ def translate_batches(
cmd = f'python3 scripts/translations/batch_translator.py "{batch_file}" --language {language_code} --api-key "{api_key}" --model {model}' cmd = f'python3 scripts/translations/batch_translator.py "{batch_file}" --language {language_code} --api-key "{api_key}" --model {model}'
try: try:
result = subprocess.run( result = subprocess.run(cmd, shell=True, capture_output=True, text=True, timeout=timeout)
cmd, shell=True, capture_output=True, text=True, timeout=timeout
)
except subprocess.TimeoutExpired: except subprocess.TimeoutExpired:
print(f"✗ Timed out after {timeout}s: {batch_file}", file=sys.stderr) print(f"✗ Timed out after {timeout}s: {batch_file}", file=sys.stderr)
return None return None
@@ -222,7 +206,7 @@ def merge_translations(translated_files, language_code):
print(f"Error: Translated file not found: {filename}") print(f"Error: Translated file not found: {filename}")
return None return None
with open(filename, "r", encoding="utf-8") as f: with open(filename, encoding="utf-8") as f:
merged.update(json.load(f)) merged.update(json.load(f))
lang_code_safe = language_code.replace("-", "_") lang_code_safe = language_code.replace("-", "_")
@@ -313,18 +297,10 @@ Examples:
) )
parser.add_argument("language", help="Language code (e.g., es-ES, de-DE, zh-CN)") parser.add_argument("language", help="Language code (e.g., es-ES, de-DE, zh-CN)")
parser.add_argument( parser.add_argument("--api-key", help="OpenAI API key (or set OPENAI_API_KEY env var)")
"--api-key", help="OpenAI API key (or set OPENAI_API_KEY env var)" parser.add_argument("--batch-size", type=int, default=500, help="Entries per batch (default: 500)")
) parser.add_argument("--no-cleanup", action="store_true", help="Keep temporary batch files")
parser.add_argument( parser.add_argument("--skip-verification", action="store_true", help="Skip final completion check")
"--batch-size", type=int, default=500, help="Entries per batch (default: 500)"
)
parser.add_argument(
"--no-cleanup", action="store_true", help="Keep temporary batch files"
)
parser.add_argument(
"--skip-verification", action="store_true", help="Skip final completion check"
)
parser.add_argument( parser.add_argument(
"--timeout", "--timeout",
type=int, type=int,
@@ -353,9 +329,7 @@ Examples:
# Verify API key # Verify API key
api_key = args.api_key or os.environ.get("OPENAI_API_KEY") api_key = args.api_key or os.environ.get("OPENAI_API_KEY")
if not api_key: if not api_key:
print( print("Error: OpenAI API key required. Provide via --api-key or OPENAI_API_KEY environment variable")
"Error: OpenAI API key required. Provide via --api-key or OPENAI_API_KEY environment variable"
)
sys.exit(1) sys.exit(1)
print("=" * 60) print("=" * 60)
@@ -369,9 +343,7 @@ Examples:
try: try:
# Step 1: Extract and split # Step 1: Extract and split
batch_files = extract_untranslated( batch_files = extract_untranslated(args.language, args.batch_size, args.include_existing)
args.language, args.batch_size, args.include_existing
)
if batch_files is None: if batch_files is None:
sys.exit(1) sys.exit(1)
scripts/translations/batch_translator.py
+12 -31
View File
@@ -9,11 +9,11 @@ Automatically translates JSON batch files to target language while preserving:
Note: Works with JSON batch files. Translation files can be TOML or JSON format. Note: Works with JSON batch files. Translation files can be TOML or JSON format.
""" """
import argparse
import json import json
import sys import sys
import argparse
from pathlib import Path
import time import time
from pathlib import Path
try: try:
from openai import OpenAI from openai import OpenAI
@@ -117,9 +117,7 @@ Return ONLY the translated JSON. No markdown, no explanations, just the JSON obj
cost_note = f", ~${cost:.4f}" if cost else "" cost_note = f", ~${cost:.4f}" if cost else ""
print(f" Tokens: {prompt_tokens:,} in / {completion_tokens:,} out{cost_note}") print(f" Tokens: {prompt_tokens:,} in / {completion_tokens:,} out{cost_note}")
def translate_batch( def translate_batch(self, batch_data: dict, target_language: str, language_code: str) -> dict:
self, batch_data: dict, target_language: str, language_code: str
) -> dict:
"""Translate a batch file using OpenAI API.""" """Translate a batch file using OpenAI API."""
# Convert batch to compact JSON for API # Convert batch to compact JSON for API
input_json = json.dumps(batch_data, ensure_ascii=False, separators=(",", ":")) input_json = json.dumps(batch_data, ensure_ascii=False, separators=(",", ":"))
@@ -134,9 +132,7 @@ Return ONLY the translated JSON. No markdown, no explanations, just the JSON obj
messages=[ messages=[
{ {
"role": "system", "role": "system",
"content": self.get_translation_prompt( "content": self.get_translation_prompt(target_language, language_code),
target_language, language_code
),
}, },
{ {
"role": "user", "role": "user",
@@ -198,9 +194,7 @@ Return ONLY the translated JSON. No markdown, no explanations, just the JSON obj
trans_placeholders = set(re.findall(placeholder_pattern, trans_value)) trans_placeholders = set(re.findall(placeholder_pattern, trans_value))
if orig_placeholders != trans_placeholders: if orig_placeholders != trans_placeholders:
issues.append( issues.append(f"Placeholder mismatch in '{key}': {orig_placeholders} vs {trans_placeholders}")
f"Placeholder mismatch in '{key}': {orig_placeholders} vs {trans_placeholders}"
)
if issues: if issues:
print("\n⚠ Validation warnings:") print("\n⚠ Validation warnings:")
@@ -276,12 +270,8 @@ Examples:
""", """,
) )
parser.add_argument( parser.add_argument("input_files", nargs="+", help="Input batch JSON file(s) or pattern")
"input_files", nargs="+", help="Input batch JSON file(s) or pattern" parser.add_argument("--api-key", help="OpenAI API key (or set OPENAI_API_KEY env var)")
)
parser.add_argument(
"--api-key", help="OpenAI API key (or set OPENAI_API_KEY env var)"
)
parser.add_argument( parser.add_argument(
"--language", "--language",
"-l", "-l",
@@ -298,9 +288,7 @@ Examples:
default="_translated", default="_translated",
help="Suffix for output files (default: _translated)", help="Suffix for output files (default: _translated)",
) )
parser.add_argument( parser.add_argument("--skip-validation", action="store_true", help="Skip validation checks")
"--skip-validation", action="store_true", help="Skip validation checks"
)
parser.add_argument( parser.add_argument(
"--delay", "--delay",
type=float, type=float,
@@ -315,9 +303,7 @@ Examples:
api_key = args.api_key or os.environ.get("OPENAI_API_KEY") api_key = args.api_key or os.environ.get("OPENAI_API_KEY")
if not api_key: if not api_key:
print( print("Error: OpenAI API key required. Provide via --api-key or OPENAI_API_KEY environment variable")
"Error: OpenAI API key required. Provide via --api-key or OPENAI_API_KEY environment variable"
)
sys.exit(1) sys.exit(1)
# Get language info # Get language info
@@ -356,13 +342,11 @@ Examples:
try: try:
# Load input file # Load input file
with open(input_file, "r", encoding="utf-8") as f: with open(input_file, encoding="utf-8") as f:
batch_data = json.load(f) batch_data = json.load(f)
# Translate # Translate
translated_data = translator.translate_batch( translated_data = translator.translate_batch(batch_data, language_name, language_code)
batch_data, language_name, language_code
)
# Validate # Validate
if not args.skip_validation: if not args.skip_validation:
@@ -396,10 +380,7 @@ Examples:
# Cost summary # Cost summary
print("-" * 60) print("-" * 60)
print( print(f"Total tokens: {translator.total_prompt_tokens:,} in / {translator.total_completion_tokens:,} out")
f"Total tokens: {translator.total_prompt_tokens:,} in / "
f"{translator.total_completion_tokens:,} out"
)
if translator.total_cost: if translator.total_cost:
print(f"Estimated cost ({args.model}): ${translator.total_cost:.4f}") print(f"Estimated cost ({args.model}): ${translator.total_cost:.4f}")
scripts/translations/bulk_auto_translate.py
+10 -21
View File
@@ -7,16 +7,13 @@ Supports concurrent translation with configurable thread pool.
import argparse import argparse
import os import os
import sys
import time
from pathlib import Path
from concurrent.futures import ThreadPoolExecutor, as_completed
import subprocess import subprocess
from typing import List, Tuple, Optional import sys
import threading import threading
import time
import tomllib import tomllib
from concurrent.futures import ThreadPoolExecutor, as_completed
from pathlib import Path
# Thread-safe print lock # Thread-safe print lock
print_lock = threading.Lock() print_lock = threading.Lock()
@@ -28,7 +25,7 @@ def safe_print(*args, **kwargs):
print(*args, **kwargs) print(*args, **kwargs)
def get_all_languages(locales_dir: Path) -> List[str]: def get_all_languages(locales_dir: Path) -> list[str]:
"""Get all language codes from locales directory.""" """Get all language codes from locales directory."""
languages = [] languages = []
@@ -45,7 +42,7 @@ def get_all_languages(locales_dir: Path) -> List[str]:
return languages return languages
def get_language_completion(locales_dir: Path, language: str) -> Optional[float]: def get_language_completion(locales_dir: Path, language: str) -> float | None:
"""Get completion percentage for a language.""" """Get completion percentage for a language."""
lang_dir = locales_dir / language lang_dir = locales_dir / language
toml_file = lang_dir / "translation.toml" toml_file = lang_dir / "translation.toml"
@@ -77,11 +74,7 @@ def get_language_completion(locales_dir: Path, language: str) -> Optional[float]
target_flat = flatten(target_data) target_flat = flatten(target_data)
# Count translated (not equal to en-US) # Count translated (not equal to en-US)
translated = sum( translated = sum(1 for k in en_us_flat if k in target_flat and target_flat[k] != en_us_flat[k])
1
for k in en_us_flat
if k in target_flat and target_flat[k] != en_us_flat[k]
)
total = len(en_us_flat) total = len(en_us_flat)
return (translated / total * 100) if total > 0 else 0.0 return (translated / total * 100) if total > 0 else 0.0
@@ -99,7 +92,7 @@ def translate_language(
skip_verification: bool, skip_verification: bool,
include_existing: bool, include_existing: bool,
model: str, model: str,
) -> Tuple[str, bool, str]: ) -> tuple[str, bool, str]:
""" """
Translate a single language. Translate a single language.
Returns: (language_code, success, message) Returns: (language_code, success, message)
@@ -178,9 +171,7 @@ Note: Requires OPENAI_API_KEY environment variable or --api-key argument.
""", """,
) )
parser.add_argument( parser.add_argument("--api-key", help="OpenAI API key (or set OPENAI_API_KEY env var)")
"--api-key", help="OpenAI API key (or set OPENAI_API_KEY env var)"
)
parser.add_argument( parser.add_argument(
"--model", "--model",
default="gpt-5.5", default="gpt-5.5",
@@ -241,9 +232,7 @@ Note: Requires OPENAI_API_KEY environment variable or --api-key argument.
# Verify API key (unless dry run) # Verify API key (unless dry run)
api_key = args.api_key or os.environ.get("OPENAI_API_KEY") api_key = args.api_key or os.environ.get("OPENAI_API_KEY")
if not args.dry_run and not api_key: if not args.dry_run and not api_key:
print( print("Error: OpenAI API key required. Provide via --api-key or OPENAI_API_KEY environment variable")
"Error: OpenAI API key required. Provide via --api-key or OPENAI_API_KEY environment variable"
)
sys.exit(1) sys.exit(1)
locales_dir = Path(args.locales_dir) locales_dir = Path(args.locales_dir)
scripts/translations/compact_translator.py
+7 -17
View File
@@ -5,11 +5,11 @@ Outputs untranslated entries in minimal JSON format with whitespace stripped.
TOML format only. TOML format only.
""" """
import argparse
import json import json
import sys import sys
from pathlib import Path
import argparse
import tomllib # Python 3.11+ (stdlib) import tomllib # Python 3.11+ (stdlib)
from pathlib import Path
class CompactTranslationExtractor: class CompactTranslationExtractor:
@@ -50,9 +50,7 @@ class CompactTranslationExtractor:
try: try:
with open(self.ignore_file, "rb") as f: with open(self.ignore_file, "rb") as f:
ignore_data = tomllib.load(f) ignore_data = tomllib.load(f)
return { return {lang: set(data.get("ignore", [])) for lang, data in ignore_data.items()}
lang: set(data.get("ignore", [])) for lang, data in ignore_data.items()
}
except Exception as e: except Exception as e:
print( print(
f"Warning: Could not load ignore file {self.ignore_file}: {e}", f"Warning: Could not load ignore file {self.ignore_file}: {e}",
@@ -60,9 +58,7 @@ class CompactTranslationExtractor:
) )
return {} return {}
def _flatten_dict( def _flatten_dict(self, d: dict, parent_key: str = "", separator: str = ".") -> dict:
self, d: dict, parent_key: str = "", separator: str = "."
) -> dict:
"""Flatten nested dictionary into dot-notation keys.""" """Flatten nested dictionary into dot-notation keys."""
items = [] items = []
for k, v in d.items(): for k, v in d.items():
@@ -102,12 +98,8 @@ class CompactTranslationExtractor:
target_value = target_flat[key] target_value = target_flat[key]
golden_value = golden_flat[key] golden_value = golden_flat[key]
if ( if (isinstance(target_value, str) and target_value.startswith("[UNTRANSLATED]")) or (
isinstance(target_value, str) golden_value == target_value and not self._is_expected_identical(key, golden_value)
and target_value.startswith("[UNTRANSLATED]")
) or (
golden_value == target_value
and not self._is_expected_identical(key, golden_value)
): ):
untranslated_keys.add(key) untranslated_keys.add(key)
@@ -151,9 +143,7 @@ def main():
default="scripts/ignore_translation.toml", default="scripts/ignore_translation.toml",
help="Path to ignore patterns file", help="Path to ignore patterns file",
) )
parser.add_argument( parser.add_argument("--max-entries", type=int, help="Maximum number of entries to output")
"--max-entries", type=int, help="Maximum number of entries to output"
)
parser.add_argument("--output", help="Output file (default: stdout)") parser.add_argument("--output", help="Output file (default: stdout)")
args = parser.parse_args() args = parser.parse_args()
scripts/translations/toml_beautifier.py
+28 -59
View File
@@ -4,13 +4,13 @@ TOML Beautifier and Structure Fixer for Stirling PDF Frontend
Restructures translation TOML files to match en-US structure and key order exactly. Restructures translation TOML files to match en-US structure and key order exactly.
""" """
import sys
from pathlib import Path
from typing import Dict, Any, List
import argparse import argparse
from collections import OrderedDict import sys
import tomllib import tomllib
from collections import OrderedDict
from pathlib import Path
from typing import Any
import tomli_w import tomli_w
@@ -20,7 +20,7 @@ class TOMLBeautifier:
self.golden_truth_file = self.locales_dir / "en-US" / "translation.toml" self.golden_truth_file = self.locales_dir / "en-US" / "translation.toml"
self.golden_structure = self._load_toml(self.golden_truth_file) self.golden_structure = self._load_toml(self.golden_truth_file)
def _load_toml(self, file_path: Path) -> Dict: def _load_toml(self, file_path: Path) -> dict:
"""Load TOML file with error handling.""" """Load TOML file with error handling."""
try: try:
with open(file_path, "rb") as f: with open(file_path, "rb") as f:
@@ -32,7 +32,7 @@ class TOMLBeautifier:
print(f"Error: Invalid TOML in {file_path}: {e}") print(f"Error: Invalid TOML in {file_path}: {e}")
sys.exit(1) sys.exit(1)
def _save_toml(self, data: Dict, file_path: Path, backup: bool = False) -> None: def _save_toml(self, data: dict, file_path: Path, backup: bool = False) -> None:
"""Save TOML file with proper formatting.""" """Save TOML file with proper formatting."""
if backup and file_path.exists(): if backup and file_path.exists():
backup_path = file_path.with_suffix(".backup.restructured.toml") backup_path = file_path.with_suffix(".backup.restructured.toml")
@@ -44,9 +44,7 @@ class TOMLBeautifier:
with open(file_path, "wb") as f: with open(file_path, "wb") as f:
tomli_w.dump(data, f) tomli_w.dump(data, f)
def _flatten_dict( def _flatten_dict(self, d: dict, parent_key: str = "", separator: str = ".") -> dict[str, Any]:
self, d: Dict, parent_key: str = "", separator: str = "."
) -> Dict[str, Any]:
"""Flatten nested dictionary into dot-notation keys.""" """Flatten nested dictionary into dot-notation keys."""
items = [] items = []
for k, v in d.items(): for k, v in d.items():
@@ -57,9 +55,7 @@ class TOMLBeautifier:
items.append((new_key, v)) items.append((new_key, v))
return dict(items) return dict(items)
def _rebuild_structure( def _rebuild_structure(self, flat_dict: dict[str, Any], reference_structure: dict) -> dict:
self, flat_dict: Dict[str, Any], reference_structure: Dict
) -> Dict:
"""Rebuild nested structure based on reference structure and available translations.""" """Rebuild nested structure based on reference structure and available translations."""
def build_recursive(ref_obj: Any, current_path: str = "") -> Any: def build_recursive(ref_obj: Any, current_path: str = "") -> Any:
@@ -94,7 +90,7 @@ class TOMLBeautifier:
return build_recursive(reference_structure) or OrderedDict() return build_recursive(reference_structure) or OrderedDict()
def restructure_translation_file(self, target_file: Path) -> Dict[str, Any]: def restructure_translation_file(self, target_file: Path) -> dict[str, Any]:
"""Restructure a translation file to match en-US structure exactly.""" """Restructure a translation file to match en-US structure exactly."""
if not target_file.exists(): if not target_file.exists():
print(f"Error: Target file does not exist: {target_file}") print(f"Error: Target file does not exist: {target_file}")
@@ -111,9 +107,7 @@ class TOMLBeautifier:
return restructured return restructured
def beautify_and_restructure( def beautify_and_restructure(self, target_file: Path, backup: bool = False) -> dict[str, Any]:
self, target_file: Path, backup: bool = False
) -> Dict[str, Any]:
"""Main function to beautify and restructure a translation file.""" """Main function to beautify and restructure a translation file."""
lang_code = target_file.parent.name lang_code = target_file.parent.name
print(f"Restructuring {lang_code} translation file...") print(f"Restructuring {lang_code} translation file...")
@@ -135,18 +129,16 @@ class TOMLBeautifier:
"language": lang_code, "language": lang_code,
"total_reference_keys": total_keys, "total_reference_keys": total_keys,
"preserved_keys": preserved_keys, "preserved_keys": preserved_keys,
"structure_match": self._compare_structures( "structure_match": self._compare_structures(self.golden_structure, restructured_data),
self.golden_structure, restructured_data
),
} }
print(f"Restructured {lang_code}: {preserved_keys}/{total_keys} keys preserved") print(f"Restructured {lang_code}: {preserved_keys}/{total_keys} keys preserved")
return result return result
def _compare_structures(self, ref: Dict, target: Dict) -> Dict[str, bool]: def _compare_structures(self, ref: dict, target: dict) -> dict[str, bool]:
"""Compare structures between reference and target.""" """Compare structures between reference and target."""
def compare_recursive(r: Any, t: Any, path: str = "") -> List[str]: def compare_recursive(r: Any, t: Any, path: str = "") -> list[str]:
issues = [] issues = []
if isinstance(r, dict) and isinstance(t, dict): if isinstance(r, dict) and isinstance(t, dict):
@@ -157,9 +149,7 @@ class TOMLBeautifier:
missing_sections = ref_keys - target_keys missing_sections = ref_keys - target_keys
if missing_sections: if missing_sections:
for section in missing_sections: for section in missing_sections:
issues.append( issues.append(f"Missing section: {path}.{section}" if path else section)
f"Missing section: {path}.{section}" if path else section
)
# Recurse into common sections # Recurse into common sections
for key in ref_keys & target_keys: for key in ref_keys & target_keys:
@@ -176,11 +166,11 @@ class TOMLBeautifier:
"total_issues": len(issues), "total_issues": len(issues),
} }
def validate_key_order(self, target_file: Path) -> Dict[str, Any]: def validate_key_order(self, target_file: Path) -> dict[str, Any]:
"""Validate that keys appear in the same order as en-US.""" """Validate that keys appear in the same order as en-US."""
target_data = self._load_toml(target_file) target_data = self._load_toml(target_file)
def get_key_order(obj: Dict, path: str = "") -> List[str]: def get_key_order(obj: dict, path: str = "") -> list[str]:
keys = [] keys = []
for key in obj.keys(): for key in obj.keys():
new_path = f"{path}.{key}" if path else key new_path = f"{path}.{key}" if path else key
@@ -195,19 +185,14 @@ class TOMLBeautifier:
# Find common keys and check their relative order # Find common keys and check their relative order
common_keys = set(golden_order) & set(target_order) common_keys = set(golden_order) & set(target_order)
golden_indices = { golden_indices = {key: idx for idx, key in enumerate(golden_order) if key in common_keys}
key: idx for idx, key in enumerate(golden_order) if key in common_keys target_indices = {key: idx for idx, key in enumerate(target_order) if key in common_keys}
}
target_indices = {
key: idx for idx, key in enumerate(target_order) if key in common_keys
}
order_preserved = all( order_preserved = all(
golden_indices[key1] < golden_indices[key2] golden_indices[key1] < golden_indices[key2]
for key1 in common_keys for key1 in common_keys
for key2 in common_keys for key2 in common_keys
if golden_indices[key1] < golden_indices[key2] if golden_indices[key1] < golden_indices[key2] and target_indices[key1] < target_indices[key2]
and target_indices[key1] < target_indices[key2]
) )
return { return {
@@ -229,12 +214,8 @@ def main():
help="Path to locales directory", help="Path to locales directory",
) )
parser.add_argument("--language", help="Restructure specific language only") parser.add_argument("--language", help="Restructure specific language only")
parser.add_argument( parser.add_argument("--all-languages", action="store_true", help="Restructure all language files")
"--all-languages", action="store_true", help="Restructure all language files" parser.add_argument("--backup", action="store_true", help="Create backup files before modifying")
)
parser.add_argument(
"--backup", action="store_true", help="Create backup files before modifying"
)
parser.add_argument( parser.add_argument(
"--validate-only", "--validate-only",
action="store_true", action="store_true",
@@ -255,21 +236,13 @@ def main():
order_result = beautifier.validate_key_order(target_file) order_result = beautifier.validate_key_order(target_file)
print(f"Key order validation for {args.language}:") print(f"Key order validation for {args.language}:")
print(f" Order preserved: {order_result['order_preserved']}") print(f" Order preserved: {order_result['order_preserved']}")
print( print(f" Common keys: {order_result['common_keys_count']}/{order_result['golden_keys_count']}")
f" Common keys: {order_result['common_keys_count']}/{order_result['golden_keys_count']}"
)
else: else:
result = beautifier.beautify_and_restructure( result = beautifier.beautify_and_restructure(target_file, backup=args.backup)
target_file, backup=args.backup
)
print(f"\nResults for {result['language']}:") print(f"\nResults for {result['language']}:")
print( print(f" Keys preserved: {result['preserved_keys']}/{result['total_reference_keys']}")
f" Keys preserved: {result['preserved_keys']}/{result['total_reference_keys']}"
)
if result["structure_match"]["total_issues"] > 0: if result["structure_match"]["total_issues"] > 0:
print( print(f" Structure issues: {result['structure_match']['total_issues']}")
f" Structure issues: {result['structure_match']['total_issues']}"
)
for issue in result["structure_match"]["issues"]: for issue in result["structure_match"]["issues"]:
print(f" - {issue}") print(f" - {issue}")
@@ -281,13 +254,9 @@ def main():
if translation_file.exists(): if translation_file.exists():
if args.validate_only: if args.validate_only:
order_result = beautifier.validate_key_order(translation_file) order_result = beautifier.validate_key_order(translation_file)
print( print(f"{lang_dir.name}: Order preserved = {order_result['order_preserved']}")
f"{lang_dir.name}: Order preserved = {order_result['order_preserved']}"
)
else: else:
result = beautifier.beautify_and_restructure( result = beautifier.beautify_and_restructure(translation_file, backup=args.backup)
translation_file, backup=args.backup
)
results.append(result) results.append(result)
if not args.validate_only and results: if not args.validate_only and results:
scripts/translations/toml_validator.py
+5 -10
View File
@@ -12,17 +12,16 @@ Usage:
python3 toml_validator.py --all-batches ar_AR python3 toml_validator.py --all-batches ar_AR
""" """
import sys
import argparse import argparse
import glob import glob
import sys
import tomllib import tomllib
def get_line_context(file_path, line_num, context_lines=3): def get_line_context(file_path, line_num, context_lines=3):
"""Get lines around the error for context""" """Get lines around the error for context"""
try: try:
with open(file_path, "r", encoding="utf-8") as f: with open(file_path, encoding="utf-8") as f:
lines = f.readlines() lines = f.readlines()
start = max(0, line_num - context_lines - 1) start = max(0, line_num - context_lines - 1)
@@ -41,7 +40,7 @@ def get_line_context(file_path, line_num, context_lines=3):
def get_character_context(file_path, char_pos, context_chars=100): def get_character_context(file_path, char_pos, context_chars=100):
"""Get characters around the error position""" """Get characters around the error position"""
try: try:
with open(file_path, "r", encoding="utf-8") as f: with open(file_path, encoding="utf-8") as f:
content = f.read() content = f.read()
start = max(0, char_pos - context_chars) start = max(0, char_pos - context_chars)
@@ -144,12 +143,8 @@ def main():
metavar="LANG", metavar="LANG",
help="Validate all batch files for a language (e.g., ar_AR)", help="Validate all batch files for a language (e.g., ar_AR)",
) )
parser.add_argument( parser.add_argument("--brief", action="store_true", help="Show brief output without context")
"--brief", action="store_true", help="Show brief output without context" parser.add_argument("--quiet", action="store_true", help="Only show files with errors")
)
parser.add_argument(
"--quiet", action="store_true", help="Only show files with errors"
)
args = parser.parse_args() args = parser.parse_args()
scripts/translations/translation_analyzer.py
+20 -42
View File
@@ -4,12 +4,11 @@ Translation Analyzer for Stirling PDF Frontend
Compares language files against en-US golden truth file. Compares language files against en-US golden truth file.
""" """
import argparse
import json import json
import sys import sys
from pathlib import Path
from typing import Dict, List, Set
import argparse
import tomllib import tomllib
from pathlib import Path
class TranslationAnalyzer: class TranslationAnalyzer:
@@ -24,7 +23,7 @@ class TranslationAnalyzer:
self.ignore_file = Path(ignore_file) self.ignore_file = Path(ignore_file)
self.ignore_patterns = self._load_ignore_patterns() self.ignore_patterns = self._load_ignore_patterns()
def _load_translation_file(self, file_path: Path) -> Dict: def _load_translation_file(self, file_path: Path) -> dict:
"""Load TOML translation file with error handling.""" """Load TOML translation file with error handling."""
try: try:
with open(file_path, "rb") as f: with open(file_path, "rb") as f:
@@ -36,7 +35,7 @@ class TranslationAnalyzer:
print(f"Error: Invalid file {file_path}: {e}") print(f"Error: Invalid file {file_path}: {e}")
sys.exit(1) sys.exit(1)
def _load_ignore_patterns(self) -> Dict[str, Set[str]]: def _load_ignore_patterns(self) -> dict[str, set[str]]:
"""Load ignore patterns from TOML file.""" """Load ignore patterns from TOML file."""
if not self.ignore_file.exists(): if not self.ignore_file.exists():
return {} return {}
@@ -56,9 +55,7 @@ class TranslationAnalyzer:
print(f"Warning: Could not load ignore file {self.ignore_file}: {e}") print(f"Warning: Could not load ignore file {self.ignore_file}: {e}")
return {} return {}
def _flatten_dict( def _flatten_dict(self, d: dict, parent_key: str = "", separator: str = ".") -> dict[str, str]:
self, d: Dict, parent_key: str = "", separator: str = "."
) -> Dict[str, str]:
"""Flatten nested dictionary into dot-notation keys.""" """Flatten nested dictionary into dot-notation keys."""
items = [] items = []
for k, v in d.items(): for k, v in d.items():
@@ -69,7 +66,7 @@ class TranslationAnalyzer:
items.append((new_key, str(v))) items.append((new_key, str(v)))
return dict(items) return dict(items)
def get_all_language_files(self) -> List[Path]: def get_all_language_files(self) -> list[Path]:
"""Get all translation files except en-US.""" """Get all translation files except en-US."""
files = [] files = []
for lang_dir in self.locales_dir.iterdir(): for lang_dir in self.locales_dir.iterdir():
@@ -79,7 +76,7 @@ class TranslationAnalyzer:
files.append(toml_file) files.append(toml_file)
return sorted(files) return sorted(files)
def find_missing_translations(self, target_file: Path) -> Set[str]: def find_missing_translations(self, target_file: Path) -> set[str]:
"""Find keys that exist in en-US but missing in target file.""" """Find keys that exist in en-US but missing in target file."""
target_data = self._load_translation_file(target_file) target_data = self._load_translation_file(target_file)
@@ -93,7 +90,7 @@ class TranslationAnalyzer:
ignore_set = self.ignore_patterns.get(lang_code, set()) ignore_set = self.ignore_patterns.get(lang_code, set())
return missing - ignore_set return missing - ignore_set
def find_untranslated_entries(self, target_file: Path) -> Set[str]: def find_untranslated_entries(self, target_file: Path) -> set[str]:
"""Find entries that appear to be untranslated (identical to en-US).""" """Find entries that appear to be untranslated (identical to en-US)."""
target_data = self._load_translation_file(target_file) target_data = self._load_translation_file(target_file)
@@ -110,10 +107,7 @@ class TranslationAnalyzer:
golden_value = golden_flat[key] golden_value = golden_flat[key]
# Check if marked as [UNTRANSLATED] or identical to en-US # Check if marked as [UNTRANSLATED] or identical to en-US
if ( if (isinstance(target_value, str) and target_value.startswith("[UNTRANSLATED]")) or (
isinstance(target_value, str)
and target_value.startswith("[UNTRANSLATED]")
) or (
golden_value == target_value golden_value == target_value
and key not in ignore_set and key not in ignore_set
and not self._is_expected_identical(key, golden_value) and not self._is_expected_identical(key, golden_value)
@@ -138,7 +132,7 @@ class TranslationAnalyzer:
return False return False
def find_extra_translations(self, target_file: Path) -> Set[str]: def find_extra_translations(self, target_file: Path) -> set[str]:
"""Find keys that exist in target file but not in en-US.""" """Find keys that exist in target file but not in en-US."""
target_data = self._load_translation_file(target_file) target_data = self._load_translation_file(target_file)
@@ -147,7 +141,7 @@ class TranslationAnalyzer:
return set(target_flat.keys()) - set(golden_flat.keys()) return set(target_flat.keys()) - set(golden_flat.keys())
def analyze_file(self, target_file: Path) -> Dict: def analyze_file(self, target_file: Path) -> dict:
"""Complete analysis of a single translation file.""" """Complete analysis of a single translation file."""
lang_code = target_file.parent.name lang_code = target_file.parent.name
@@ -172,14 +166,10 @@ class TranslationAnalyzer:
if key in target_flat: if key in target_flat:
value = target_flat[key] value = target_flat[key]
if not (isinstance(value, str) and value.startswith("[UNTRANSLATED]")): if not (isinstance(value, str) and value.startswith("[UNTRANSLATED]")):
if ( if key not in untranslated: # Not identical to en-US (unless expected)
key not in untranslated
): # Not identical to en-US (unless expected)
properly_translated += 1 properly_translated += 1
completion_rate = ( completion_rate = (properly_translated / total_keys) * 100 if total_keys > 0 else 0
(properly_translated / total_keys) * 100 if total_keys > 0 else 0
)
return { return {
"language": lang_code, "language": lang_code,
@@ -194,7 +184,7 @@ class TranslationAnalyzer:
"completion_rate": completion_rate, "completion_rate": completion_rate,
} }
def analyze_all_files(self) -> List[Dict]: def analyze_all_files(self) -> list[dict]:
"""Analyze all translation files.""" """Analyze all translation files."""
results = [] results = []
for file_path in self.get_all_language_files(): for file_path in self.get_all_language_files():
@@ -203,9 +193,7 @@ class TranslationAnalyzer:
def main(): def main():
parser = argparse.ArgumentParser( parser = argparse.ArgumentParser(description="Analyze translation files against en-US golden truth")
description="Analyze translation files against en-US golden truth"
)
parser.add_argument( parser.add_argument(
"--locales-dir", "--locales-dir",
default="frontend/editor/public/locales", default="frontend/editor/public/locales",
@@ -217,20 +205,14 @@ def main():
help="Path to ignore patterns TOML file", help="Path to ignore patterns TOML file",
) )
parser.add_argument("--language", help="Analyze specific language only") parser.add_argument("--language", help="Analyze specific language only")
parser.add_argument( parser.add_argument("--missing-only", action="store_true", help="Show only missing translations")
"--missing-only", action="store_true", help="Show only missing translations"
)
parser.add_argument( parser.add_argument(
"--untranslated-only", "--untranslated-only",
action="store_true", action="store_true",
help="Show only untranslated entries", help="Show only untranslated entries",
) )
parser.add_argument( parser.add_argument("--summary", action="store_true", help="Show summary statistics only")
"--summary", action="store_true", help="Show summary statistics only" parser.add_argument("--format", choices=["text", "json"], default="text", help="Output format")
)
parser.add_argument(
"--format", choices=["text", "json"], default="text", help="Output format"
)
args = parser.parse_args() args = parser.parse_args()
@@ -287,16 +269,12 @@ def main():
print(f"\n{'=' * 60}") print(f"\n{'=' * 60}")
print("SUMMARY") print("SUMMARY")
print(f"{'=' * 60}") print(f"{'=' * 60}")
avg_completion = ( avg_completion = sum(r["completion_rate"] for r in results) / len(results) if results else 0
sum(r["completion_rate"] for r in results) / len(results) if results else 0
)
print(f"Average Completion Rate: {avg_completion:.1f}%") print(f"Average Completion Rate: {avg_completion:.1f}%")
print(f"Languages Analyzed: {len(results)}") print(f"Languages Analyzed: {len(results)}")
# Top languages by completion # Top languages by completion
sorted_by_completion = sorted( sorted_by_completion = sorted(results, key=lambda x: x["completion_rate"], reverse=True)
results, key=lambda x: x["completion_rate"], reverse=True
)
print("\nTop 5 Most Complete Languages:") print("\nTop 5 Most Complete Languages:")
for result in sorted_by_completion[:5]: for result in sorted_by_completion[:5]:
print(f" {result['language']}: {result['completion_rate']:.1f}%") print(f" {result['language']}: {result['completion_rate']:.1f}%")
scripts/translations/translation_merger.py
+29 -83
View File
@@ -6,28 +6,24 @@ Useful for AI-assisted translation workflows.
TOML format only. TOML format only.
""" """
import os
import argparse import argparse
import json import json
import os
import shutil import shutil
import sys import sys
import tomllib
from datetime import datetime from datetime import datetime
from pathlib import Path from pathlib import Path
from typing import Any from typing import Any
import tomllib
import tomli_w import tomli_w
class TranslationMerger: class TranslationMerger:
def __init__( def __init__(
self, self,
locales_dir: str = os.path.join( locales_dir: str = os.path.join(os.getcwd(), "frontend", "editor", "public", "locales"),
os.getcwd(), "frontend", "editor", "public", "locales" ignore_file: str = os.path.join(os.getcwd(), "scripts", "ignore_translation.toml"),
),
ignore_file: str = os.path.join(
os.getcwd(), "scripts", "ignore_translation.toml"
),
): ):
self.locales_dir = Path(locales_dir) self.locales_dir = Path(locales_dir)
self.golden_truth_file = self.locales_dir / "en-US" / "translation.toml" self.golden_truth_file = self.locales_dir / "en-US" / "translation.toml"
@@ -47,14 +43,10 @@ class TranslationMerger:
print(f"Error: Invalid file {file_path}: {e}") print(f"Error: Invalid file {file_path}: {e}")
sys.exit(1) sys.exit(1)
def _save_translation_file( def _save_translation_file(self, data: dict[str, Any], file_path: Path, backup: bool = False) -> None:
self, data: dict[str, Any], file_path: Path, backup: bool = False
) -> None:
"""Save TOML translation file with backup option.""" """Save TOML translation file with backup option."""
if backup and file_path.exists(): if backup and file_path.exists():
backup_path = file_path.with_suffix( backup_path = file_path.with_suffix(f".backup.{datetime.now().strftime('%Y%m%d_%H%M%S')}.toml")
f".backup.{datetime.now().strftime('%Y%m%d_%H%M%S')}.toml"
)
shutil.copy2(file_path, backup_path) shutil.copy2(file_path, backup_path)
print(f"Backup created: {backup_path}") print(f"Backup created: {backup_path}")
@@ -71,9 +63,7 @@ class TranslationMerger:
ignore_data = tomllib.load(f) ignore_data = tomllib.load(f)
# Convert to sets for faster lookup # Convert to sets for faster lookup
return { return {lang: set(data.get("ignore", [])) for lang, data in ignore_data.items()}
lang: set(data.get("ignore", [])) for lang, data in ignore_data.items()
}
except Exception as e: except Exception as e:
print(f"Warning: Could not load ignore file {self.ignore_file}: {e}") print(f"Warning: Could not load ignore file {self.ignore_file}: {e}")
return {} return {}
@@ -89,9 +79,7 @@ class TranslationMerger:
return None return None
return current return current
def _set_nested_value( def _set_nested_value(self, data: dict[str, Any], key_path: str, value: Any) -> None:
self, data: dict[str, Any], key_path: str, value: Any
) -> None:
"""Set value in nested dict using dot notation.""" """Set value in nested dict using dot notation."""
keys = key_path.split(".") keys = key_path.split(".")
current = data current = data
@@ -101,16 +89,12 @@ class TranslationMerger:
elif not isinstance(current[key], dict): elif not isinstance(current[key], dict):
# If the current value is not a dict, we can't nest into it # If the current value is not a dict, we can't nest into it
# This handles cases where a key exists as a string but we need to make it a dict # This handles cases where a key exists as a string but we need to make it a dict
print( print(f"Warning: Converting non-dict value at '{key}' to dict to allow nesting")
f"Warning: Converting non-dict value at '{key}' to dict to allow nesting"
)
current[key] = {} current[key] = {}
current = current[key] current = current[key]
current[keys[-1]] = value current[keys[-1]] = value
def _flatten_dict( def _flatten_dict(self, d: dict[str, Any], parent_key: str = "", separator: str = ".") -> dict[str, Any]:
self, d: dict[str, Any], parent_key: str = "", separator: str = "."
) -> dict[str, Any]:
"""Flatten nested dictionary into dot-notation keys.""" """Flatten nested dictionary into dot-notation keys."""
items = [] items = []
for k, v in d.items(): for k, v in d.items():
@@ -207,9 +191,7 @@ class TranslationMerger:
"data": target_data, "data": target_data,
} }
def extract_untranslated_entries( def extract_untranslated_entries(self, target_file: Path, output_file: Path | None = None) -> dict[str, Any]:
self, target_file: Path, output_file: Path | None = None
) -> dict[str, Any]:
"""Extract entries marked as untranslated or identical to en-US for AI translation.""" """Extract entries marked as untranslated or identical to en-US for AI translation."""
if not target_file.exists(): if not target_file.exists():
print(f"Error: Target file does not exist: {target_file}") print(f"Error: Target file does not exist: {target_file}")
@@ -233,9 +215,7 @@ class TranslationMerger:
"reason": "marked_untranslated", "reason": "marked_untranslated",
} }
# Check if identical to golden (and should be translated) # Check if identical to golden (and should be translated)
elif value == golden_value and not self._is_expected_identical( elif value == golden_value and not self._is_expected_identical(key, value):
key, value
):
untranslated_entries[key] = { untranslated_entries[key] = {
"original": golden_value, "original": golden_value,
"current": value, "current": value,
@@ -279,9 +259,7 @@ class TranslationMerger:
for key, translation in translations.items(): for key, translation in translations.items():
try: try:
# Remove [UNTRANSLATED] marker if present # Remove [UNTRANSLATED] marker if present
if isinstance(translation, str) and translation.startswith( if isinstance(translation, str) and translation.startswith("[UNTRANSLATED]"):
"[UNTRANSLATED]"
):
translation = translation.replace("[UNTRANSLATED]", "").strip() translation = translation.replace("[UNTRANSLATED]", "").strip()
self._set_nested_value(target_data, key, translation) self._set_nested_value(target_data, key, translation)
@@ -390,45 +368,25 @@ def main():
subparsers = parser.add_subparsers(dest="command", help="Available commands") subparsers = parser.add_subparsers(dest="command", help="Available commands")
# Add missing command # Add missing command
add_parser = subparsers.add_parser( add_parser = subparsers.add_parser("add-missing", help="Add missing translations from en-US")
"add-missing", help="Add missing translations from en-US" add_parser.add_argument("--backup", action="store_true", help="Create backup before modifying files")
)
add_parser.add_argument(
"--backup", action="store_true", help="Create backup before modifying files"
)
# Extract untranslated command # Extract untranslated command
extract_parser = subparsers.add_parser( extract_parser = subparsers.add_parser("extract-untranslated", help="Extract untranslated entries")
"extract-untranslated", help="Extract untranslated entries"
)
extract_parser.add_argument("--output", help="Output file path") extract_parser.add_argument("--output", help="Output file path")
# Create template command # Create template command
template_parser = subparsers.add_parser( template_parser = subparsers.add_parser("create-template", help="Create AI translation template")
"create-template", help="Create AI translation template" template_parser.add_argument("--output", required=True, help="Output template file path")
)
template_parser.add_argument(
"--output", required=True, help="Output template file path"
)
# Apply translations command # Apply translations command
apply_parser = subparsers.add_parser( apply_parser = subparsers.add_parser("apply-translations", help="Apply translations from JSON file")
"apply-translations", help="Apply translations from JSON file" apply_parser.add_argument("--translations-file", required=True, help="JSON file with translations")
) apply_parser.add_argument("--backup", action="store_true", help="Create backup before modifying files")
apply_parser.add_argument(
"--translations-file", required=True, help="JSON file with translations"
)
apply_parser.add_argument(
"--backup", action="store_true", help="Create backup before modifying files"
)
# Remove unused translations command # Remove unused translations command
remove_parser = subparsers.add_parser( remove_parser = subparsers.add_parser("remove-unused", help="Remove unused translations not present in en-US")
"remove-unused", help="Remove unused translations not present in en-US" remove_parser.add_argument("--backup", action="store_true", help="Create backup before modifying files")
)
remove_parser.add_argument(
"--backup", action="store_true", help="Create backup before modifying files"
)
args = parser.parse_args() args = parser.parse_args()
@@ -453,9 +411,7 @@ def main():
continue continue
target_file = lang_dir / "translation.toml" target_file = lang_dir / "translation.toml"
print(f"Processing {lang_dir.name}...") print(f"Processing {lang_dir.name}...")
result = merger.add_missing_translations( result = merger.add_missing_translations(target_file, backup=args.backup)
target_file, backup=args.backup
)
added = result["added_count"] added = result["added_count"]
total_added += added total_added += added
print(f"Added {added} missing translations") print(f"Added {added} missing translations")
@@ -475,9 +431,7 @@ def main():
continue continue
target_file = lang_dir / "translation.toml" target_file = lang_dir / "translation.toml"
print(f"Processing {lang_dir.name}...") print(f"Processing {lang_dir.name}...")
result = merger.remove_unused_translations( result = merger.remove_unused_translations(target_file, backup=args.backup)
target_file, backup=args.backup
)
removed = result["removed_count"] removed = result["removed_count"]
total_removed += removed total_removed += removed
print(f"Removed {removed} unused translations") print(f"Removed {removed} unused translations")
@@ -489,11 +443,7 @@ def main():
sys.exit(1) sys.exit(1)
lang_dir = Path(args.locales_dir) / args.language lang_dir = Path(args.locales_dir) / args.language
target_file = lang_dir / "translation.toml" target_file = lang_dir / "translation.toml"
output_file = ( output_file = Path(args.output) if args.output else target_file.with_suffix(".untranslated.json")
Path(args.output)
if args.output
else target_file.with_suffix(".untranslated.json")
)
untranslated = merger.extract_untranslated_entries(target_file, output_file) untranslated = merger.extract_untranslated_entries(target_file, output_file)
print(f"Extracted {len(untranslated)} untranslated entries to {output_file}") print(f"Extracted {len(untranslated)} untranslated entries to {output_file}")
@@ -512,22 +462,18 @@ def main():
lang_dir = Path(args.locales_dir) / args.language lang_dir = Path(args.locales_dir) / args.language
target_file = lang_dir / "translation.toml" target_file = lang_dir / "translation.toml"
with open(args.translations_file, "r", encoding="utf-8") as f: with open(args.translations_file, encoding="utf-8") as f:
translations_data = json.load(f) translations_data = json.load(f)
# Extract translations from template format or simple dict # Extract translations from template format or simple dict
if "translations" in translations_data: if "translations" in translations_data:
translations = { translations = {
k: v["translated"] k: v["translated"] for k, v in translations_data["translations"].items() if v.get("translated")
for k, v in translations_data["translations"].items()
if v.get("translated")
} }
else: else:
translations = translations_data translations = translations_data
result = merger.apply_translations( result = merger.apply_translations(target_file, translations, backup=args.backup)
target_file, translations, backup=args.backup
)
if result["success"]: if result["success"]:
print(f"Applied {result['applied_count']} translations") print(f"Applied {result['applied_count']} translations")
scripts/translations/validate_json_structure.py
+9 -20
View File
@@ -13,15 +13,14 @@ Usage:
python scripts/translations/validate_json_structure.py [--language LANG] python scripts/translations/validate_json_structure.py [--language LANG]
""" """
import argparse
import json import json
import sys import sys
from pathlib import Path
from typing import Dict, Set
import argparse
import tomllib # Python 3.11+ (stdlib) import tomllib # Python 3.11+ (stdlib)
from pathlib import Path
def get_all_keys(d: dict, parent_key: str = "", sep: str = ".") -> Set[str]: def get_all_keys(d: dict, parent_key: str = "", sep: str = ".") -> set[str]:
"""Get all keys from nested dict as dot-notation paths.""" """Get all keys from nested dict as dot-notation paths."""
keys = set() keys = set()
for k, v in d.items(): for k, v in d.items():
@@ -42,9 +41,7 @@ def validate_translation_file(file_path: Path) -> tuple[bool, str]:
return False, f"Error reading file: {str(e)}" return False, f"Error reading file: {str(e)}"
def validate_structure( def validate_structure(en_us_keys: set[str], lang_keys: set[str], lang_code: str) -> dict:
en_us_keys: Set[str], lang_keys: Set[str], lang_code: str
) -> Dict:
"""Compare structure between en-US and target language.""" """Compare structure between en-US and target language."""
missing_keys = en_us_keys - lang_keys missing_keys = en_us_keys - lang_keys
extra_keys = lang_keys - en_us_keys extra_keys = lang_keys - en_us_keys
@@ -60,7 +57,7 @@ def validate_structure(
} }
def print_validation_result(result: Dict, verbose: bool = False): def print_validation_result(result: dict, verbose: bool = False):
"""Print validation results in readable format.""" """Print validation results in readable format."""
lang = result["language"] lang = result["language"]
@@ -111,9 +108,7 @@ def main():
help="Specific language code to validate (e.g., es-ES)", help="Specific language code to validate (e.g., es-ES)",
default=None, default=None,
) )
parser.add_argument( parser.add_argument("--verbose", "-v", action="store_true", help="Show all missing/extra keys")
"--verbose", "-v", action="store_true", help="Show all missing/extra keys"
)
parser.add_argument("--json", action="store_true", help="Output results as JSON") parser.add_argument("--json", action="store_true", help="Output results as JSON")
args = parser.parse_args() args = parser.parse_args()
@@ -162,9 +157,7 @@ def main():
# First check if file is valid # First check if file is valid
is_valid, message = validate_translation_file(lang_path) is_valid, message = validate_translation_file(lang_path)
if not is_valid: if not is_valid:
json_errors.append( json_errors.append({"language": lang_code, "file": str(lang_path), "error": message})
{"language": lang_code, "file": str(lang_path), "error": message}
)
continue continue
# Load and compare structure # Load and compare structure
@@ -194,9 +187,7 @@ def main():
print("\n📊 Structure Validation Summary:") print("\n📊 Structure Validation Summary:")
print(f" Languages validated: {len(results)}") print(f" Languages validated: {len(results)}")
perfect = sum( perfect = sum(1 for r in results if r["missing_count"] == 0 and r["extra_count"] == 0)
1 for r in results if r["missing_count"] == 0 and r["extra_count"] == 0
)
print(f" Perfect matches: {perfect}/{len(results)}") print(f" Perfect matches: {perfect}/{len(results)}")
total_missing = sum(r["missing_count"] for r in results) total_missing = sum(r["missing_count"] for r in results)
@@ -211,9 +202,7 @@ def main():
print("\n✅ All translations have perfect structure!") print("\n✅ All translations have perfect structure!")
# Exit with error code if issues found # Exit with error code if issues found
has_issues = len(json_errors) > 0 or any( has_issues = len(json_errors) > 0 or any(r["missing_count"] > 0 or r["extra_count"] > 0 for r in results)
r["missing_count"] > 0 or r["extra_count"] > 0 for r in results
)
sys.exit(1 if has_issues else 0) sys.exit(1 if has_issues else 0)
scripts/translations/validate_placeholders.py
+7 -12
View File
@@ -9,23 +9,22 @@ Usage:
--fix: Automatically remove extra placeholders (use with caution) --fix: Automatically remove extra placeholders (use with caution)
""" """
import argparse
import json import json
import re import re
import sys import sys
from pathlib import Path
from typing import Dict, List, Set
import argparse
import tomllib # Python 3.11+ (stdlib) import tomllib # Python 3.11+ (stdlib)
from pathlib import Path
def find_placeholders(text: str) -> Set[str]: def find_placeholders(text: str) -> set[str]:
"""Find all placeholders in text like {n}, {{var}}, {0}, etc.""" """Find all placeholders in text like {n}, {{var}}, {0}, etc."""
if not isinstance(text, str): if not isinstance(text, str):
return set() return set()
return set(re.findall(r"\{\{?[^}]+\}\}?", text)) return set(re.findall(r"\{\{?[^}]+\}\}?", text))
def flatten_dict(d: dict, parent_key: str = "", sep: str = ".") -> Dict[str, str]: def flatten_dict(d: dict, parent_key: str = "", sep: str = ".") -> dict[str, str]:
"""Flatten nested dict to dot-notation keys.""" """Flatten nested dict to dot-notation keys."""
items = [] items = []
for k, v in d.items(): for k, v in d.items():
@@ -37,9 +36,7 @@ def flatten_dict(d: dict, parent_key: str = "", sep: str = ".") -> Dict[str, str
return dict(items) return dict(items)
def validate_language( def validate_language(en_us_flat: dict[str, str], lang_flat: dict[str, str], lang_code: str) -> list[dict]:
en_us_flat: Dict[str, str], lang_flat: Dict[str, str], lang_code: str
) -> List[Dict]:
"""Validate placeholders for a language against en-US.""" """Validate placeholders for a language against en-US."""
issues = [] issues = []
@@ -67,7 +64,7 @@ def validate_language(
return issues return issues
def print_issues(issues: List[Dict], verbose: bool = False): def print_issues(issues: list[dict], verbose: bool = False):
"""Print validation issues in a readable format.""" """Print validation issues in a readable format."""
if not issues: if not issues:
print("✅ No placeholder validation issues found!") print("✅ No placeholder validation issues found!")
@@ -93,9 +90,7 @@ def print_issues(issues: List[Dict], verbose: bool = False):
def main(): def main():
parser = argparse.ArgumentParser( parser = argparse.ArgumentParser(description="Validate translation placeholder consistency")
description="Validate translation placeholder consistency"
)
parser.add_argument( parser.add_argument(
"--language", "--language",
help="Specific language code to validate (e.g., es-ES)", help="Specific language code to validate (e.g., es-ES)",
scripts/type3_to_cff.py
+36 -53
View File
@@ -19,9 +19,9 @@ import argparse
import json import json
import math import math
import sys import sys
from collections.abc import Iterable, Sequence
from dataclasses import dataclass from dataclasses import dataclass
from pathlib import Path from pathlib import Path
from typing import Dict, Iterable, List, Optional, Sequence, Tuple
from fontTools.fontBuilder import FontBuilder from fontTools.fontBuilder import FontBuilder
from fontTools.misc.fixedTools import otRound from fontTools.misc.fixedTools import otRound
@@ -29,17 +29,16 @@ from fontTools.pens.cu2quPen import Cu2QuPen
from fontTools.pens.t2CharStringPen import T2CharStringPen from fontTools.pens.t2CharStringPen import T2CharStringPen
from fontTools.pens.ttGlyphPen import TTGlyphPen from fontTools.pens.ttGlyphPen import TTGlyphPen
Command = dict[str, object]
Command = Dict[str, object] Matrix = tuple[float, float, float, float, float, float]
Matrix = Tuple[float, float, float, float, float, float]
@dataclass @dataclass
class GlyphSource: class GlyphSource:
name: str name: str
width: float width: float
unicode: Optional[int] unicode: int | None
char_code: Optional[int] char_code: int | None
outline: Sequence[Command] outline: Sequence[Command]
@@ -48,34 +47,20 @@ class GlyphBuildResult:
name: str name: str
width: int width: int
charstring: object charstring: object
ttf_glyph: Optional[object] ttf_glyph: object | None
unicode: Optional[int] unicode: int | None
char_code: Optional[int] char_code: int | None
bounds: Optional[Tuple[float, float, float, float]] bounds: tuple[float, float, float, float] | None
def parse_args() -> argparse.Namespace: def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser( parser = argparse.ArgumentParser(description="Synthesize fonts from Type3 glyph JSON.")
description="Synthesize fonts from Type3 glyph JSON." parser.add_argument("--input", required=True, help="Path to glyph JSON emitted by the backend")
) parser.add_argument("--otf-output", required=True, help="Destination path for the CFF/OTF font")
parser.add_argument( parser.add_argument("--ttf-output", help="Optional destination path for a TrueType font")
"--input", required=True, help="Path to glyph JSON emitted by the backend" parser.add_argument("--family-name", default="Type3 Synth", help="Family name for the output")
) parser.add_argument("--style-name", default="Regular", help="Style name for the output")
parser.add_argument( parser.add_argument("--units-per-em", type=int, default=1000, help="Units per EM value")
"--otf-output", required=True, help="Destination path for the CFF/OTF font"
)
parser.add_argument(
"--ttf-output", help="Optional destination path for a TrueType font"
)
parser.add_argument(
"--family-name", default="Type3 Synth", help="Family name for the output"
)
parser.add_argument(
"--style-name", default="Regular", help="Style name for the output"
)
parser.add_argument(
"--units-per-em", type=int, default=1000, help="Units per EM value"
)
parser.add_argument( parser.add_argument(
"--cu2qu-error", "--cu2qu-error",
type=float, type=float,
@@ -85,7 +70,7 @@ def parse_args() -> argparse.Namespace:
return parser.parse_args() return parser.parse_args()
def load_json(path: Path) -> Dict[str, object]: def load_json(path: Path) -> dict[str, object]:
try: try:
with path.open("r", encoding="utf-8") as handle: with path.open("r", encoding="utf-8") as handle:
return json.load(handle) return json.load(handle)
@@ -94,7 +79,7 @@ def load_json(path: Path) -> Dict[str, object]:
sys.exit(2) sys.exit(2)
def parse_font_matrix(rows: Optional[Iterable[Iterable[float]]]) -> Matrix: def parse_font_matrix(rows: Iterable[Iterable[float]] | None) -> Matrix:
""" """
Retrieve the raw 2×3 FontMatrix entries for diagnostics. Type3 glyph Retrieve the raw 2×3 FontMatrix entries for diagnostics. Type3 glyph
outlines in our extractor are emitted in their native coordinate system, so outlines in our extractor are emitted in their native coordinate system, so
@@ -102,7 +87,7 @@ def parse_font_matrix(rows: Optional[Iterable[Iterable[float]]]) -> Matrix:
""" """
if not rows: if not rows:
return (1.0, 0.0, 0.0, 1.0, 0.0, 0.0) return (1.0, 0.0, 0.0, 1.0, 0.0, 0.0)
values: List[List[float]] = [] values: list[list[float]] = []
for row in rows: for row in rows:
try: try:
values.append([float(col) for col in row]) values.append([float(col) for col in row])
@@ -132,10 +117,10 @@ def resolve_width(raw_width: float, default: int) -> int:
def quadratic_to_cubic( def quadratic_to_cubic(
current: Tuple[float, float], current: tuple[float, float],
ctrl: Tuple[float, float], ctrl: tuple[float, float],
end: Tuple[float, float], end: tuple[float, float],
) -> Tuple[Tuple[float, float], Tuple[float, float], Tuple[float, float]]: ) -> tuple[tuple[float, float], tuple[float, float], tuple[float, float]]:
""" """
Convert a quadratic Bézier segment to cubic control points. Convert a quadratic Bézier segment to cubic control points.
""" """
@@ -150,9 +135,9 @@ def quadratic_to_cubic(
return c1, c2, end return c1, c2, end
def iterate_glyphs(data: Dict[str, object]) -> List[GlyphSource]: def iterate_glyphs(data: dict[str, object]) -> list[GlyphSource]:
glyph_records = data.get("glyphs") or [] glyph_records = data.get("glyphs") or []
sources: List[GlyphSource] = [] sources: list[GlyphSource] = []
for index, record in enumerate(glyph_records, start=1): for index, record in enumerate(glyph_records, start=1):
if not isinstance(record, dict): if not isinstance(record, dict):
continue continue
@@ -170,9 +155,7 @@ def iterate_glyphs(data: Dict[str, object]) -> List[GlyphSource]:
char_code_value = record.get("code") char_code_value = record.get("code")
if not isinstance(char_code_value, int): if not isinstance(char_code_value, int):
char_code_value = record.get("charCodeRaw") char_code_value = record.get("charCodeRaw")
if not isinstance(char_code_value, int) or not ( if not isinstance(char_code_value, int) or not (0 <= char_code_value <= 0x10FFFF):
0 <= char_code_value <= 0x10FFFF
):
char_code_value = None char_code_value = None
outline = record.get("outline") outline = record.get("outline")
if not isinstance(outline, list): if not isinstance(outline, list):
@@ -192,19 +175,19 @@ def iterate_glyphs(data: Dict[str, object]) -> List[GlyphSource]:
def build_cff_charstring( def build_cff_charstring(
glyph: GlyphSource, glyph: GlyphSource,
width: int, width: int,
) -> Tuple[object, Optional[Tuple[float, float, float, float]]]: ) -> tuple[object, tuple[float, float, float, float] | None]:
pen = T2CharStringPen(width=width, glyphSet=None) pen = T2CharStringPen(width=width, glyphSet=None)
bounds = [math.inf, math.inf, -math.inf, -math.inf] bounds = [math.inf, math.inf, -math.inf, -math.inf]
def update_bounds(point: Tuple[float, float]) -> None: def update_bounds(point: tuple[float, float]) -> None:
x, y = point x, y = point
bounds[0] = min(bounds[0], x) bounds[0] = min(bounds[0], x)
bounds[1] = min(bounds[1], y) bounds[1] = min(bounds[1], y)
bounds[2] = max(bounds[2], x) bounds[2] = max(bounds[2], x)
bounds[3] = max(bounds[3], y) bounds[3] = max(bounds[3], y)
current: Optional[Tuple[float, float]] = None current: tuple[float, float] | None = None
start_point: Optional[Tuple[float, float]] = None start_point: tuple[float, float] | None = None
open_path = False open_path = False
for command in glyph.outline: for command in glyph.outline:
@@ -278,7 +261,7 @@ def build_cff_charstring(
return charstring, bbox return charstring, bbox
def build_ttf_glyph(glyph: GlyphSource, max_error: float) -> Optional[object]: def build_ttf_glyph(glyph: GlyphSource, max_error: float) -> object | None:
pen = TTGlyphPen(glyphSet=None) pen = TTGlyphPen(glyphSet=None)
draw_pen = Cu2QuPen(pen, max_error, reverse_direction=False) draw_pen = Cu2QuPen(pen, max_error, reverse_direction=False)
@@ -321,9 +304,9 @@ def build_ttf_glyph(glyph: GlyphSource, max_error: float) -> Optional[object]:
def synthesise_fonts( def synthesise_fonts(
data: Dict[str, object], data: dict[str, object],
otf_output: Path, otf_output: Path,
ttf_output: Optional[Path], ttf_output: Path | None,
family_name: str, family_name: str,
style_name: str, style_name: str,
units_per_em: int, units_per_em: int,
@@ -332,7 +315,7 @@ def synthesise_fonts(
_font_matrix = parse_font_matrix(data.get("fontMatrix")) _font_matrix = parse_font_matrix(data.get("fontMatrix"))
glyphs = iterate_glyphs(data) glyphs = iterate_glyphs(data)
results: List[GlyphBuildResult] = [] results: list[GlyphBuildResult] = []
global_y_min = math.inf global_y_min = math.inf
global_y_max = -math.inf global_y_max = -math.inf
@@ -377,7 +360,7 @@ def synthesise_fonts(
horizontal_metrics = {result.name: (result.width, 0) for result in results} horizontal_metrics = {result.name: (result.width, 0) for result in results}
horizontal_metrics[".notdef"] = (default_width, 0) horizontal_metrics[".notdef"] = (default_width, 0)
cmap: Dict[int, str] = {} cmap: dict[int, str] = {}
next_private = 0xF000 next_private = 0xF000
for result in results: for result in results:
code_point = result.unicode code_point = result.unicode
@@ -433,7 +416,7 @@ def synthesise_fonts(
if ttf_output is None: if ttf_output is None:
return return
glyph_objects: Dict[str, object] = {} glyph_objects: dict[str, object] = {}
empty_pen = TTGlyphPen(None) empty_pen = TTGlyphPen(None)
empty_pen.moveTo((0, 0)) empty_pen.moveTo((0, 0))
empty_pen.lineTo((0, 0)) empty_pen.lineTo((0, 0))
scripts/update_type3_library.py
+14 -32
View File
@@ -17,25 +17,15 @@ from __future__ import annotations
import argparse import argparse
import json import json
import sys import sys
from collections.abc import Iterable
from pathlib import Path from pathlib import Path
from typing import Dict, Iterable, List, Optional, Tuple
REPO_ROOT = Path(__file__).resolve().parents[1] REPO_ROOT = Path(__file__).resolve().parents[1]
DEFAULT_SIGNATURES = REPO_ROOT / "docs" / "type3" / "signatures" DEFAULT_SIGNATURES = REPO_ROOT / "docs" / "type3" / "signatures"
DEFAULT_INDEX = ( DEFAULT_INDEX = REPO_ROOT / "app" / "core" / "src" / "main" / "resources" / "type3" / "library" / "index.json"
REPO_ROOT
/ "app"
/ "core"
/ "src"
/ "main"
/ "resources"
/ "type3"
/ "library"
/ "index.json"
)
def normalize_alias(value: Optional[str]) -> Optional[str]: def normalize_alias(value: str | None) -> str | None:
if not value: if not value:
return None return None
trimmed = value.strip() trimmed = value.strip()
@@ -75,9 +65,9 @@ def iter_signature_fonts(signature_file: Path):
} }
def make_alias_index(entries: List[Dict]) -> Tuple[Dict[str, Dict], Dict[str, Dict]]: def make_alias_index(entries: list[dict]) -> tuple[dict[str, dict], dict[str, dict]]:
alias_index: Dict[str, Dict] = {} alias_index: dict[str, dict] = {}
signature_index: Dict[str, Dict] = {} signature_index: dict[str, dict] = {}
for entry in entries: for entry in entries:
for alias in entry.get("aliases", []) or []: for alias in entry.get("aliases", []) or []:
normalized = normalize_alias(alias) normalized = normalize_alias(alias)
@@ -91,7 +81,7 @@ def make_alias_index(entries: List[Dict]) -> Tuple[Dict[str, Dict], Dict[str, Di
return alias_index, signature_index return alias_index, signature_index
def ensure_list(container: Dict, key: str) -> List: def ensure_list(container: dict, key: str) -> list:
value = container.get(key) value = container.get(key)
if isinstance(value, list): if isinstance(value, list):
return value return value
@@ -100,11 +90,11 @@ def ensure_list(container: Dict, key: str) -> List:
return value return value
def merge_sorted_unique(values: Iterable[int]) -> List[int]: def merge_sorted_unique(values: Iterable[int]) -> list[int]:
return sorted({int(v) for v in values if isinstance(v, int)}) return sorted({int(v) for v in values if isinstance(v, int)})
def normalize_source_path(pdf_path: Optional[str]) -> Optional[str]: def normalize_source_path(pdf_path: str | None) -> str | None:
if not pdf_path: if not pdf_path:
return None return None
try: try:
@@ -117,13 +107,13 @@ def normalize_source_path(pdf_path: Optional[str]) -> Optional[str]:
def update_library( def update_library(
signatures_dir: Path, index_path: Path, apply_changes: bool signatures_dir: Path, index_path: Path, apply_changes: bool
) -> Tuple[int, int, List[Tuple[str, Path]]]: ) -> tuple[int, int, list[tuple[str, Path]]]:
entries = load_json(index_path) entries = load_json(index_path)
alias_index, signature_index = make_alias_index(entries) alias_index, signature_index = make_alias_index(entries)
modifications = 0 modifications = 0
updated_entries = set() updated_entries = set()
unmatched: List[Tuple[str, Path]] = [] unmatched: list[tuple[str, Path]] = []
signature_files = sorted(signatures_dir.glob("*.json")) signature_files = sorted(signatures_dir.glob("*.json"))
if not signature_files: if not signature_files:
@@ -198,9 +188,7 @@ def update_library(
def parse_args() -> argparse.Namespace: def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser( parser = argparse.ArgumentParser(description="Update Type3 library index using signature dumps.")
description="Update Type3 library index using signature dumps."
)
parser.add_argument( parser.add_argument(
"--signatures-dir", "--signatures-dir",
type=Path, type=Path,
@@ -223,11 +211,7 @@ def parse_args() -> argparse.Namespace:
def main() -> None: def main() -> None:
args = parse_args() args = parse_args()
signatures_dir = ( signatures_dir = args.signatures_dir if args.signatures_dir.is_absolute() else (REPO_ROOT / args.signatures_dir)
args.signatures_dir
if args.signatures_dir.is_absolute()
else (REPO_ROOT / args.signatures_dir)
)
index_path = args.index if args.index.is_absolute() else (REPO_ROOT / args.index) index_path = args.index if args.index.is_absolute() else (REPO_ROOT / args.index)
if not signatures_dir.exists(): if not signatures_dir.exists():
@@ -237,9 +221,7 @@ def main() -> None:
print(f"Index file not found: {index_path}", file=sys.stderr) print(f"Index file not found: {index_path}", file=sys.stderr)
sys.exit(2) sys.exit(2)
modifications, updated_entries, unmatched = update_library( modifications, updated_entries, unmatched = update_library(signatures_dir, index_path, apply_changes=args.apply)
signatures_dir, index_path, apply_changes=args.apply
)
mode = "APPLIED" if args.apply else "DRY-RUN" mode = "APPLIED" if args.apply else "DRY-RUN"
print( print(
testing/compose/run-multinode-regression.sh
+3 -7
View File
@@ -25,12 +25,8 @@ elif [ "$SEED" = 1 ]; then
fi fi
echo "==> Checking Python + behave..." echo "==> Checking Python + behave..."
PY="${PYTHON:-python}" if ! uv run --project ../../engine --locked --group cucumber python -c "import behave" 2>/dev/null; then
command -v "$PY" >/dev/null || PY=python3 echo " could not load the central uv cucumber environment"; exit 1
if ! "$PY" -c "import behave" 2>/dev/null; then
echo " installing test deps..."
"$PY" -m pip install -q -r "$CUKE_DIR/requirements.txt" || {
echo " could not install behave; install $CUKE_DIR/requirements.txt manually"; exit 1; }
fi fi
REPORT_DIR="$(pwd)/multinode/regression-report" REPORT_DIR="$(pwd)/multinode/regression-report"
@@ -39,7 +35,7 @@ mkdir -p "$REPORT_DIR"
run_behave() { # $1=tags $2=label run_behave() { # $1=tags $2=label
echo "==> behave features/multinode --tags='$1' ($2)" echo "==> behave features/multinode --tags='$1' ($2)"
# behave.ini excludes features/multinode by default; -e here overrides that while still excluding the licence-gated enterprise suite. # behave.ini excludes features/multinode by default; -e here overrides that while still excluding the licence-gated enterprise suite.
( cd "$CUKE_DIR" && "$PY" -m behave features/multinode -e "features/enterprise" \ ( cd "$CUKE_DIR" && uv run --project ../../engine --locked --group cucumber python -m behave features/multinode -e "features/enterprise" \
--tags="$1" --no-capture --format plain --format html --outfile "$REPORT_DIR/$2.html" ) --tags="$1" --no-capture --format plain --format html --outfile "$REPORT_DIR/$2.html" )
return $? return $?
} }
testing/cucumber/requirements.in
-7
View File
@@ -1,7 +0,0 @@
behave
behave-html-formatter
requests
pypdf
reportlab
PyCryptodome
qrcode[pil]
testing/cucumber/requirements.txt
-305
View File
@@ -1,305 +0,0 @@
#
# This file is autogenerated by pip-compile with Python 3.13
# by the following command:
#
# pip-compile --generate-hashes --output-file='testing\cucumber\requirements.txt' --strip-extras 'testing\cucumber\requirements.in'
#
behave==1.3.3 \
--hash=sha256:2b8f4b64ed2ea756a5a2a73e23defc1c4631e9e724c499e46661778453ebaf51 \
--hash=sha256:89bdb62af8fb9f147ce245736a5de69f025e5edfb66f1fbe16c5007493f842c0
# via
# -r testing/cucumber/requirements.in
# behave-html-formatter
behave-html-formatter==0.9.10 \
--hash=sha256:c5a9ad3edcac7be5766b14aacce46794885c749c4741fc93f8fc3a4bf2a891aa \
--hash=sha256:fff7ac2118463701423645ad5a12636845bfd6c8a2dd52097b524b4f290aa7c8
# via -r testing/cucumber/requirements.in
certifi==2026.6.17 \
--hash=sha256:024c88eeec92ca068db80f02b8b07c9cef7b9fe261d1d535abfd5abd6f6af432 \
--hash=sha256:2227dcbaafe0d2f59279d1762ddddc37783ed4354594f194ffc31d20f41fc3db
# via requests
charset-normalizer==3.4.9 \
--hash=sha256:0327fcd59a935777d83410750c50600ee9571af2846f71ce40f25b13da1ef380 \
--hash=sha256:03d07803992c6c7bbc976327f34b18b6160327fc81cb82c9d504720ac0be3b62 \
--hash=sha256:04ce310cb89c15df659582aee80a0603788732a5e017d5bd5c81158106ce249c \
--hash=sha256:0d861473f743244d349b50f850d10eb87aeb22bbdcc8e64f79273c94af5a8226 \
--hash=sha256:0e94703ec9684807f20cfb5eed95c70f67f2a8f21ad620146d7b5a13677b93e5 \
--hash=sha256:0fa1aec2d32bcc03c8fa0f6f1712caad1adc38509f31142112e5c9daf5b9c833 \
--hash=sha256:16b65ea0f2465b6fb52aa22de5eca612aa964ddfec00a912e26f4656cbef890b \
--hash=sha256:16d10d789dd9bcca1173c95af82c58433122564b7bc39385124be735a35cbe99 \
--hash=sha256:19ac87f93086ce37b86e098888555c4b4bc48102279bae3350098c0ed664b501 \
--hash=sha256:1d22856ffbe153a602df38e4a5464f0b748a54002e0d69ac6d2ad0a197cc99ec \
--hash=sha256:21e764fd1e70b6a3e205a0e46f3051701f98a8cb3fad66eeb80e48bb502f8698 \
--hash=sha256:231ddcbb35e2ff8973e1365db41fe0572662893b99a05deb183b68ad4c0c8bd4 \
--hash=sha256:253a4a220747e8b5faf57ec320c4f5efb0cef05f647420bf267143ec15dba10a \
--hash=sha256:280081916dc341820640489a66e4696049401ef1cf6dd672f672e70ad915aca3 \
--hash=sha256:2a441ea71902098ffe78c5abe6c494f44160b4af614ed16c3d9a3b1d17fd8ee2 \
--hash=sha256:304b13570067b2547562e308af560b3963857b1fa90bd6afd978130130fe2d6a \
--hash=sha256:32286a2c8d167e897177b673176c1e3e00d4057caf5d2b64eef9a3666b03018e \
--hash=sha256:33bdcc2a32c0a0e861f60841a512c8acc658c87c2ac59d89e3a46dacf7d866e4 \
--hash=sha256:375b83ed0aecfce76c16d198fbc21f3b11b337d68662bea0a995046682a11419 \
--hash=sha256:3c09a49d6cde137258beb3d551994a2927fd35ad5cf96aed573f61bbd67c5f84 \
--hash=sha256:3d92613ec25e43b05f042302531ec0f00b8445190e43325880cbd6ab7c2581da \
--hash=sha256:40a126142a56b2dfc0aacbad1de8310cbf60da7656db0e6b16eebd48e3e93519 \
--hash=sha256:416c229f77e5ea25b3dfd4b582f8d73d7e43c22320302b9ab128a2d3a0b38efe \
--hash=sha256:432786d3561e69aeeae6c7e8648964ce0ad05736120135601f87ac26b9c83381 \
--hash=sha256:43b9e366a31fdd1c87d0eb08f579b4a82b723ea54338f040d6b4e518a026ea29 \
--hash=sha256:440eede837960000d74978f0eba527be106b5b9aee0daf779d395276ed0b0614 \
--hash=sha256:45b0cc4e3556cd875e09102988d1ab8356c998b596c9fced84547c8138b487a0 \
--hash=sha256:476743fe6dfe14a2da12e3ac79125dc84a3b2cf8094369a47a1529b0cd8549fe \
--hash=sha256:4773092f8019072343a7447203308b176e10199920eb02d6195e81bbb3274c29 \
--hash=sha256:4b3dac63058cc36820b0dd072f89898604e2d39686fe05321729d00d8ac185a0 \
--hash=sha256:4d1c96a7a18b9690a4d46df09e3e3382406ae3213727cd1019ebade1c4a81917 \
--hash=sha256:51307f5c71007673a2bf8232ad973483d281e74cb99c8c5a990af1eefa6277d9 \
--hash=sha256:51447e9aa2684679af07ca5021c3db526e0284347ebf4ffcec1154c3350cfe32 \
--hash=sha256:58150c9f9b9a552505912d182ccdf26f6396fb6094816ceebcbb20eecabaed94 \
--hash=sha256:5b10cd92fc5c498b35a8635df6d5a100207f88b63a4dc1de7ef9a548e1e2cd63 \
--hash=sha256:5e226f6218febc71f6c1fc2fafb91c226f75bdc1d8fb12d66823716e891608fd \
--hash=sha256:609b3ba8fcc0fb5ab7af00719d0fb6ad0cb518e48e7712d12fd68f1327951198 \
--hash=sha256:60f44ade2cf573dad7a277e6f8ca9a51a21dda572b13bd7d8539bb3cd5dbedde \
--hash=sha256:611057cc5d5c0afc743ba8be6bd828c17e0aaa8643f9d0a9b9bb7dea80eb8012 \
--hash=sha256:6366a16e1a25018694d6a5d784d09b046edc9eac40ea2b54065c3052672516a1 \
--hash=sha256:65a7ff3f705e57d392f7261b6d0550fe137c3019477431f1c355e0db0a7d3e15 \
--hash=sha256:673611bbd43f0810bec0b0f028ddeaaa501190339cac411f347ac76917c3ae7b \
--hash=sha256:67830fc78e67501f47bb950471b2dcb9b35b140084429318e862895a8e89c993 \
--hash=sha256:68ce9f4d6b26d5ccbf7fd4459bf75f74a0a146677ebba80597df60cbdb20e6f4 \
--hash=sha256:68e5f26a1ad57ded6d1cfb85331d1c1a195314756471d97758c48498bb4dcdf5 \
--hash=sha256:69b157c5d3292bcd443faca052f3096f637f1e074b98212a933c074ae23dc3b8 \
--hash=sha256:75286256590a6320cf106a0d28970d3560aad9ee09aa7b34fb40524792436d35 \
--hash=sha256:78841cccf1af7b40f6f716338d50c0902dbe88d9f800b3c973b7a9a0a693a642 \
--hash=sha256:78fa18e436a1a0e58dbd7e02fc4473f3f32cceb12df9dfca542d075961c307d2 \
--hash=sha256:79580094b00d1789d1f93ea55bc43cb2f611910c72235b7657f3482ddcc1b22d \
--hash=sha256:7b86a2b16095d250c6f58b3d9b2eee6f4147754344f3dab0922f7c9bf7d226c9 \
--hash=sha256:83aed2c10721ddd90f68140685391b50811a880af20654c59af6b6c66c40513c \
--hash=sha256:84fd18bcc17526fc2b3c1af7d2b9217d32c9c04448c16ec693b9b4f1985c3d33 \
--hash=sha256:871ff67ea1aad4dfd91736464934d56b32dac49f9fbe16cddba36198a7b3a0db \
--hash=sha256:898f0e9068ca27d37f8e83a5b962821df851532e6c4a7d615c1c033f9da6eedf \
--hash=sha256:8a79d9f4d8001473a30c163556b3c3bfebec837495a412dde78b51672f6134f9 \
--hash=sha256:8c041122946b7ba21bb32c45b1aa57b1be35527690aeb3c5c234521085632eee \
--hash=sha256:90c44bc373b7687f6948b693cceaea1348ae0975d7474746559494468e3c1d84 \
--hash=sha256:9104ed0bd76a429d46f9ec0dbc9b08ad1d2dcdf2b00a5a0daa1c145329b35b44 \
--hash=sha256:920079c3f7456fa213e0829ed2073aaa727fd39d889ead5b4f35d0de5460d04f \
--hash=sha256:93d59d504b230e83c7a843251681959a0b6a9cd76f6e146ce1b8a80eb8739af9 \
--hash=sha256:9b2aff1c7b3884512b9512c3eaadd9bab39fb45042ffaaa1dd08ff2b9f8109d9 \
--hash=sha256:9b8e0f3107e2200b76f6054de99016eac3ee6762713587b36baaa7e4bd2ae177 \
--hash=sha256:9bb41182d93ea91f60b4bc8fbf4c820c69ef8a12ab2d917f3f1834f1acad07e8 \
--hash=sha256:9cdef90ae47919cae358d8ab15797a800ed41da7aba5d72419fb510729e2ed4b \
--hash=sha256:a1786910334ed46ab1dd73222f2cd1e05c2c3bb39f6dddb4f8b36fc382058a39 \
--hash=sha256:a4cfde78a9f2880208d16a93b795726a3017d5977e08d1e162a7a31322479c41 \
--hash=sha256:a4fbdde9dd4a9ce5fd52c2b3a347bb50cc89483ef783f1cb00d408c13f7a96c0 \
--hash=sha256:aa99adc8f081b475a12843953db36831eaf83ec33eb46a90629ca6a5de45a616 \
--hash=sha256:ac351b3b8014eead140e77e9717e2992c6bbe30b63bc3422422eb84865412e3d \
--hash=sha256:ad41ba96094304aa090f5a30cb6e4fb3b3f1c264c523394b4c39bbacc4dc92ba \
--hash=sha256:b5314963fce9b0b12743891de876e724997864ee22aa496f903f426c7e2fa5b2 \
--hash=sha256:bcf74c1df76758a395bf0af608c04c82257523f55c9868b334f06270d0f2112b \
--hash=sha256:bd47ba7fc3ca94896759ea0109775132d3e7ab921fbf54038e1bab2e46c313c9 \
--hash=sha256:c0323c9daef75ef2e5083624b4585018a0c9d5e3b40f607eed81a311270b934b \
--hash=sha256:c1225416b463483160e4af85d5fc3a9690ccb53fd4b1865a6437825f5ede3209 \
--hash=sha256:c1c948747b03be832dceed96ca815cef7360de9aa19d37c730f8e3f6101aca48 \
--hash=sha256:c25fe15c70c59eb7c5ce8c06a1f3fa1da0ecc5ea1e7a5922c40fd2fa9b0d5046 \
--hash=sha256:cc1b0fff8ead343dae06305f954eb8468ba0ec1a97881f42489d198e4ce3c632 \
--hash=sha256:cd6280cf040f233bd7d3407b743b4b4c74f70e8e1c4199cb112a62c941c0772a \
--hash=sha256:cd6c3d4b783c556fa00bf540854e42f135e2f256abd29669fcd0da0f2dec79c2 \
--hash=sha256:d4d6fcde76f94f5cb9e43e9e9a61f16dacefd228cbbf6f1a09bd9b219a92f1a1 \
--hash=sha256:ddf4af30b417d9fe16481e9b81c27ab2a7cde1ff7ba3e85653b02db7d145dc7b \
--hash=sha256:df115d4d83168fdf2cae48ef1ff6d1cb4c466364e30861b37121de0f3bf1b990 \
--hash=sha256:df7276909358e5635ae203673ab7e509ddd224225a8d6b0790bf13eb2bde1cc5 \
--hash=sha256:e4fd89cc178bced6ad29cb3e6dd4aa63fa5017c3524dbd0b25998fb64a87cc8b \
--hash=sha256:e9701d0049d92c16703a42771b98d560b95248949f23f8cf7b4eddd201814fb9 \
--hash=sha256:ee2f2a527e3c1a6e6411eb4209642e138b544a2d72fe5d0d76daf77b24063534 \
--hash=sha256:f7fb7d750cfa0a070d2c24e831fd3481019a60dd317ea2b39acbcebc08b6ed81 \
--hash=sha256:f840ed6d8ecba8255df8c42b87fadeda98ddfc6eeec05e2dc66e26d46dd6f58a \
--hash=sha256:f86c6358749bd4fda175388691e3ba8c46e24c5347d0afd20f9b7edfc9faf07d \
--hash=sha256:fa36ec09ef71d158186bc79e359ff5fdd6e7996fe8ab638f00d6b93139ba4fcf \
--hash=sha256:fe2c7201c642b7c308f1675355ad7ff7b66acfe3541625efe5a3ad38f29d6115
# via
# reportlab
# requests
colorama==0.4.6 \
--hash=sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44 \
--hash=sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6
# via
# behave
# qrcode
cucumber-expressions==20.0.0 \
--hash=sha256:5cbd4012c66584aa82ada990a6e7cb131274796e132e905d27d95ae9a2ca0f48 \
--hash=sha256:8a0434529efd7ca6e2052934ec8d677c7e24edc0fad3b1d1b1bc4bbca5e521f3
# via behave
cucumber-tag-expressions==10.0.0 \
--hash=sha256:496f4a834e4e7ef9c1ae3f6f9c22f237a32f1916f96c32d8b8f900865f988f81 \
--hash=sha256:b3ca4163660b247760031f88797ecde2c5384aac85f9d5908cb4750eddde89b7
# via behave
idna==3.18 \
--hash=sha256:7f952cbe720b688055e3f87de14f5c3e5fdaa8bc3928985c4077ca689de849a2 \
--hash=sha256:ffb385a7e039654cef1ab9ef32c6fafe283c0c0467bba1d9029738ce4a14a848
# via requests
parse==1.22.1 \
--hash=sha256:20f0925a46f06602485ac90d751764d0697fd8455aaa97489ba8953a4b66de32 \
--hash=sha256:d3a4740ec3da338e2b258b2d69741b731eadfddca59e24a14bc4ee5fce38c911
# via
# behave
# parse-type
parse-type==0.6.6 \
--hash=sha256:3ca79bbe71e170dfccc8ec6c341edfd1c2a0fc1e5cfd18330f93af938de2348c \
--hash=sha256:513a3784104839770d690e04339a8b4d33439fcd5dd99f2e4580f9fc1097bfb2
# via behave
pillow==12.3.0 \
--hash=sha256:00808c5e14ef63ac5161091d242999076604ff74b883423a11e5d7bbb38bf756 \
--hash=sha256:04f01d28a6aaff387bf842a13be313df23ba0597a44f1a976c9feb3c6ff4711a \
--hash=sha256:06ff022112bc9cbf83b60f8e028d94ad87b60621706487e65f673de61610ab59 \
--hash=sha256:0740a512dc522224c77d9aa5a8d70d8b7d73fb91f2c21125d8d025d3b8990e45 \
--hash=sha256:0847a763afefb695bc912d7c131e7e0632d4edc1d8698f58ddabec8e46b8b6d3 \
--hash=sha256:0dd2064cbc55aaec028ef5fbb60fa47bb6c3e7918e07ff17935284b227a9d2df \
--hash=sha256:0feb2e9d6ad6c9e3c06effe9d00f3f1e618a6643273576b016f591e9315a7139 \
--hash=sha256:10e41f0fbf1eec8cfd234b8fe17a4caac7c9d0db4c204d3c173a8f9f6ef3232b \
--hash=sha256:1182d52bc2d5e5d7d0949503aa7e36d12f42205dc287e4883f407b1988820d39 \
--hash=sha256:164b31cd1a0490ab6efae01aa5df49da7061be0af1b30e035b6e9a1bfe34ee6e \
--hash=sha256:1657923d2d45afb66526e5b933e5b3052e6bdea196c90d3abb2424e18c77dae8 \
--hash=sha256:186941b6aef820ad110fb01fb06eb925374dc3a21b17e37ec9a53b250c6fe2d1 \
--hash=sha256:1cca606cd25738df4ed873d5ad46bbdb3d83b5cbca291f6b4ff13a4df6b0bbe8 \
--hash=sha256:21900ce7ba264168cd50defae43cd75d25c833ad4ad6e73ffc5596d12e25ac89 \
--hash=sha256:236ff70b9312fb68943c703aa842ca6a758abfa45ac187a5e7c1452e96ef72b5 \
--hash=sha256:23aceaa007d6172b02c277f0cd359c79492bbb14f7072b4ede9fbcaf20648130 \
--hash=sha256:23d27a3e0307ec2244cc51e7287b919aa68d097504ebe19df4e76a98a3eea5bd \
--hash=sha256:24870b09b224f7ae3c39ed07d10e819d06f8720bc551847b1d623832b5b0e28d \
--hash=sha256:251bf95b67017e27b13d82f5b326234ca62d70f9cf4c2b9032de2358a3b12c7b \
--hash=sha256:25b9b82bb22e6e2b3cd07b39c68b7b862001226cb3dff7130d1cb914121b39ed \
--hash=sha256:28ce87c5ab450a9dd970b52e5aca5fe63ed432d18a2eaddd1979a00a1ba24ace \
--hash=sha256:300557495eb45ebb8aec96c2da9c4be642fbf7cd937278b4013ba894ea8eb0eb \
--hash=sha256:30f2aa603c41533cc25c05acd0da21636e84a315768feb631c937177db558931 \
--hash=sha256:331b624368d4f1d069149002f25f44bc61c8919ce8ddb3c45bdad8f6e2d89510 \
--hash=sha256:37d6d0a00072fd2948eb22bce7e1475f34569d90c87c59f7a2ec59541b77f7a6 \
--hash=sha256:37dc8f7bbb66efe481bb60defacef820c950c24713fb44962ed6aa2a50966de1 \
--hash=sha256:3b8182a766685eaa002637e28b4ec8d6b18819a0c71f579bf0dbaa5830297cce \
--hash=sha256:3edce1d53195db527e0191f84b71d02022de0540bf43a16ed734ed7537b07385 \
--hash=sha256:446c34dcc4324b084a53b705127dc15717b22c5e140ae0a3c38349d4efec071e \
--hash=sha256:4998562bf62a445225f22e07c896bb04b35b1b1f2eb6d760584c9c51d7a5f78c \
--hash=sha256:4b0a7fe987b14c31ebda6083f74f22b561fd3739bc0ac51e019622e3d72668c7 \
--hash=sha256:4e8c2a84d977f50b9daed6eeaf3baef67d00d5d74d932288f02cb94518ee3ace \
--hash=sha256:4f883547d4b7f0495ebe7056b0cc2aea76094e7a4abc8e933540f3271df27d9c \
--hash=sha256:514435a37670e3e5e08f3945b68718b6ed329bb84367777e16f9f4dfe1e61a0f \
--hash=sha256:53aa02d20d10c3d814d536aa4e5ac9b84ca0ff5a88377963b085ad6822f93e64 \
--hash=sha256:5594fc43d548a7ed94949d139aa1341b270f1863f11cfd37f5a6c8b778a6b67f \
--hash=sha256:571b9fcb07b97ef3a492028fb3d2dc0993ca23a06138b0315286566d29ef718a \
--hash=sha256:57b3d78c95ba9059768b10e28b813002261d3f3dfc55cc48b0c988f625175827 \
--hash=sha256:5afb51d599ea772b8365ae807ae557f18bccfe46ab261fd1c2a9ed700fc6eb17 \
--hash=sha256:6b02afb9b97f65fbca5f31db6a2a3ba21aa93030225f150fa3f249717e938fb4 \
--hash=sha256:6c0016e7b354317c4e9e525b937ac8596c38d2d232b419529b9cd7a1cd46e39a \
--hash=sha256:71d6097b330eea8fd15097780c8e89cb1a8ce7838669f48c5bacd6f663dd4701 \
--hash=sha256:756c768d0c9c2955feb7a56c37ea24aea2e369f8d36a88da270b6a9f19e62b5e \
--hash=sha256:78cb2c6865a35ab8ff8b75fd122f6033b92a62c82801110e48ddd6c936a45d91 \
--hash=sha256:7a743ff716f746fc19a9557f60dab1600d4613255f8a7aeb3cdde4db7eb15a66 \
--hash=sha256:85f998ea1848bc6757289e739cfbdda3a04adfd58b02fc018ce54d754a5ce468 \
--hash=sha256:8728f216dcdb6e6d555cf971cb34076139ad74b31fc2c14da4fafc741c5f6217 \
--hash=sha256:877c3f311ff35410f690861c4409e7ccbf0cd2f878e50628a28e5a0bb689e658 \
--hash=sha256:8cd2f7bdda092d99c9fc2fb7391354f306d01443d22785d0cbfafa2e2c8bb418 \
--hash=sha256:8e95e1385e4998ae9694eeaa4730ba5457ff61185b3a55e2e7bea0880aef452a \
--hash=sha256:962864dc93511324d51ddbb5b9f8731bf71675b93ca612a07441896f4688fb8c \
--hash=sha256:9cf95fe4d0f84c82d282745d9bb08ad9f926efa00be4697e767b814ce40d4330 \
--hash=sha256:9e881fca225083806662a5c43d627d215f258ff43c890f831966c7d7ba9c7402 \
--hash=sha256:a2b55dd6b2a4c4b7d87ffa56bdb33fdc5fdb9a462173861a7bc097f17d91cb09 \
--hash=sha256:a45650e8ce7fafffd731db8550230db6b0d306d181a90b67d3e6bca2f1990930 \
--hash=sha256:a876864214e136f0eb367788dbd7df045f4806801518e2cfe9e13229cfe06d8f \
--hash=sha256:ae26d61dfa7a47befdc7572b521024e8745f3d809bd95ca9505a7bba9ef849ec \
--hash=sha256:af8d94b0db561cf68b88a267c5c44b49e134f525d0dc2cb7ed413a66bc23559a \
--hash=sha256:b343699e8308bdc51978310e1c959c584e7869cc8c40780058c87da7781a1e94 \
--hash=sha256:b3c777e849237620b022f7f297dd67705f9f5cf1685f09f02e46f93e92725468 \
--hash=sha256:b629de27fda84b42cde7edef0d85f13b958b47f6e9bbcbba9b673c562a89bd8b \
--hash=sha256:ba09209fbe443b4acccebe845d8a138b89a8f4fbaeedd44953490b5315d5e965 \
--hash=sha256:ba54cfebe86920a559a7c4d6b9050791c20513650a1952ebe3368c7dc70306f8 \
--hash=sha256:bcb46e2f9feff8d06323983bd83ed00c201fdcab3d74973e7072a889b3979fcd \
--hash=sha256:bcc33feacfaefce60c12fd500a277533bdc02b10a19f7f6d348763d8140bbba7 \
--hash=sha256:bf16ba1b4d0b6b7c8e534936632270cf70eb00dbe09005bc345b2677b726855c \
--hash=sha256:cf1845d02ad822a369a49f2bb9345b1614744267682e7a03527dc3bf6eea1777 \
--hash=sha256:d69141514cc30b774ceea5e3ed3a6635c8d8a96edf664689b890f4089111fb35 \
--hash=sha256:d9c7f76c0673154f044e9d78c8655fb4213f6ca31a836df48b40fe5d187717b9 \
--hash=sha256:dbce0b29841537a2fa4a214c2bbf14de3587c9680caa9b4e217568472490b28f \
--hash=sha256:dc624f6bc473dacdf7ef7eb8678d0d08edf15cd94fad6ae5c7d6cc67a4e4902f \
--hash=sha256:e158cb00350dc278f3b91551101aa7d12415a66ebf2c91d8d5ac14e56ddd3ad0 \
--hash=sha256:e491916b378fba47242221bb9ead245211b70d504f495d105d17b14a24b4907c \
--hash=sha256:e795b7eb908249c4e43c7c99fac7c2c75dab0c43566e37db472a355f63693d71 \
--hash=sha256:e7e480451b9fa137494bccd3a7d69adbe8ac65a87d97be61e11f1b1050a5bac3 \
--hash=sha256:e91206ee562682b51b98ef4b26a6ef48fd84e15fd4c4bc5ec768eb641d206838 \
--hash=sha256:e9871b1ffbfa9656b60aeee92ed5136a5742696006fa322b29ea3d8da0ecc9cf \
--hash=sha256:e9aeb04d6aef139de265b29683e119b638208f88cf73cdd1658aa07221165321 \
--hash=sha256:ebaea975e03d3141d9d3a507df75c9b3ec90fa9d2ffd07567b3a978d9d790b26 \
--hash=sha256:f0606c8bf2cdefea14a43530f7657cbbb7ecf1c4222512492ef4a4434a9501ec \
--hash=sha256:f13c32a3abd6079a66d9526e18dad9b6d280384d49d7c54040cd57b6424041d9 \
--hash=sha256:f7401aebd7f581d7f83a439d87d474999317ee099218e5ad25d125290990ba65 \
--hash=sha256:fa4ecea169a355be7a3ade2c783e2ed12f0e40d2c5621cda8b3297faf7fbb9f5 \
--hash=sha256:fbd139c8447d25dd750ab79ee274cc5e1fe80fc56340ab10b18a195e1b6eca3e \
--hash=sha256:fdafc9cce40277e0f7a0feabce0ee50dd2fa1800f3b38015e51296b5e814048d \
--hash=sha256:fe3cca2e4e8a592be0f269a1ca4835c25199d9f3ce815c8491048f785b0a0198 \
--hash=sha256:ffd0c5368496f41b0944be820fcb7a838aa6e623d250b01acf2643939c3f99d7
# via
# qrcode
# reportlab
pycryptodome==3.23.0 \
--hash=sha256:0011f7f00cdb74879142011f95133274741778abba114ceca229adbf8e62c3e4 \
--hash=sha256:11eeeb6917903876f134b56ba11abe95c0b0fd5e3330def218083c7d98bbcb3c \
--hash=sha256:14e15c081e912c4b0d75632acd8382dfce45b258667aa3c67caf7a4d4c13f630 \
--hash=sha256:156df9667ad9f2ad26255926524e1c136d6664b741547deb0a86a9acf5ea631f \
--hash=sha256:187058ab80b3281b1de11c2e6842a357a1f71b42cb1e15bce373f3d238135c27 \
--hash=sha256:257bb3572c63ad8ba40b89f6fc9d63a2a628e9f9708d31ee26560925ebe0210a \
--hash=sha256:350ebc1eba1da729b35ab7627a833a1a355ee4e852d8ba0447fafe7b14504d56 \
--hash=sha256:447700a657182d60338bab09fdb27518f8856aecd80ae4c6bdddb67ff5da44ef \
--hash=sha256:45c69ad715ca1a94f778215a11e66b7ff989d792a4d63b68dc586a1da1392ff5 \
--hash=sha256:4764e64b269fc83b00f682c47443c2e6e85b18273712b98aa43bcb77f8570477 \
--hash=sha256:507dbead45474b62b2bbe318eb1c4c8ee641077532067fec9c1aa82c31f84886 \
--hash=sha256:53ecbafc2b55353edcebd64bf5da94a2a2cdf5090a6915bcca6eca6cc452585a \
--hash=sha256:573a0b3017e06f2cffd27d92ef22e46aa3be87a2d317a5abf7cc0e84e321bd75 \
--hash=sha256:63dad881b99ca653302b2c7191998dd677226222a3f2ea79999aa51ce695f720 \
--hash=sha256:64093fc334c1eccfd3933c134c4457c34eaca235eeae49d69449dc4728079339 \
--hash=sha256:6501790c5b62a29fcb227bd6b62012181d886a767ce9ed03b303d1f22eb5c625 \
--hash=sha256:67bd81fcbe34f43ad9422ee8fd4843c8e7198dd88dd3d40e6de42ee65fbe1490 \
--hash=sha256:6fe8258e2039eceb74dfec66b3672552b6b7d2c235b2dfecc05d16b8921649a8 \
--hash=sha256:763d1d74f56f031788e5d307029caef067febf890cd1f8bf61183ae142f1a77b \
--hash=sha256:7ac1080a8da569bde76c0a104589c4f414b8ba296c0b3738cf39a466a9fb1818 \
--hash=sha256:865d83c906b0fc6a59b510deceee656b6bc1c4fa0d82176e2b77e97a420a996a \
--hash=sha256:89d4d56153efc4d81defe8b65fd0821ef8b2d5ddf8ed19df31ba2f00872b8002 \
--hash=sha256:90460fc9e088ce095f9ee8356722d4f10f86e5be06e2354230a9880b9c549aae \
--hash=sha256:93837e379a3e5fd2bb00302a47aee9fdf7940d83595be3915752c74033d17ca7 \
--hash=sha256:954af0e2bd7cea83ce72243b14e4fb518b18f0c1649b576d114973e2073b273d \
--hash=sha256:9a53a4fe5cb075075d515797d6ce2f56772ea7e6a1e5e4b96cf78a14bac3d265 \
--hash=sha256:9a77627a330ab23ca43b48b130e202582e91cc69619947840ea4d2d1be21eb39 \
--hash=sha256:a176b79c49af27d7f6c12e4b178b0824626f40a7b9fed08f712291b6d54bf566 \
--hash=sha256:a7fc76bf273353dc7e5207d172b83f569540fc9a28d63171061c42e361d22353 \
--hash=sha256:aa0698f65e5b570426fc31b8162ed4603b0c2841cbb9088e2b01641e3065915b \
--hash=sha256:b34e8e11d97889df57166eda1e1ddd7676da5fcd4d71a0062a760e75060514b4 \
--hash=sha256:c75b52aacc6c0c260f204cbdd834f76edc9fb0d8e0da9fbf8352ef58202564e2 \
--hash=sha256:c8987bd3307a39bc03df5c8e0e3d8be0c4c3518b7f044b0f4c15d1aa78f52575 \
--hash=sha256:ce64e84a962b63a47a592690bdc16a7eaf709d2c2697ababf24a0def566899a6 \
--hash=sha256:cfb5cd445280c5b0a4e6187a7ce8de5a07b5f3f897f235caa11f1f435f182843 \
--hash=sha256:d8e95564beb8782abfd9e431c974e14563a794a4944c29d6d3b7b5ea042110b4 \
--hash=sha256:d97618c9c6684a97ef7637ba43bdf6663a2e2e77efe0f863cce97a76af396446 \
--hash=sha256:ddb95b49df036ddd264a0ad246d1be5b672000f12d6961ea2c267083a5e19379 \
--hash=sha256:dea827b4d55ee390dc89b2afe5927d4308a8b538ae91d9c6f7a5090f397af1aa \
--hash=sha256:e3f2d0aaf8080bda0587d58fc9fe4766e012441e2eed4269a77de6aea981c8be \
--hash=sha256:eb8f24adb74984aa0e5d07a2368ad95276cf38051fe2dc6605cbcf482e04f2a7
# via -r testing/cucumber/requirements.in
pypdf==6.14.2 \
--hash=sha256:3f07891af76dc002657e04993ab9b4de81de29f9013b9761d0b7968bff12e946 \
--hash=sha256:7873f502fe4385e79539b21d872392dc0c4e3714327c15881cbc7fbfd1f95b25
# via -r testing/cucumber/requirements.in
qrcode==8.2 \
--hash=sha256:16e64e0716c14960108e85d853062c9e8bba5ca8252c0b4d0231b9df4060ff4f \
--hash=sha256:35c3f2a4172b33136ab9f6b3ef1c00260dd2f66f858f24d88418a015f446506c
# via -r testing/cucumber/requirements.in
reportlab==5.0.0 \
--hash=sha256:9d5a3affa84919e1111ede580031266a570e93b1ce388219621347965ff1d93c \
--hash=sha256:e4494a0c6623ae213bb856fba523171b2b54a7bf629fda02d5e525a7b899a784
# via -r testing/cucumber/requirements.in
requests==2.34.2 \
--hash=sha256:2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0 \
--hash=sha256:f288924cae4e29463698d6d60bc6a4da69c89185ad1e0bcc4104f584e960b9ed
# via -r testing/cucumber/requirements.in
six==1.17.0 \
--hash=sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274 \
--hash=sha256:ff70335d468e7eb6ec65b95b99d3a2836546063f63acc5171de367e834932a81
# via
# behave
# parse-type
urllib3==2.7.0 \
--hash=sha256:231e0ec3b63ceb14667c67be60f2f2c40a518cb38b03af60abc813da26505f4c \
--hash=sha256:9fb4c81ebbb1ce9531cce37674bbc6f1360472bc18ca9a553ede278ef7276897
# via requests
testing/test.sh
+1 -1
View File
@@ -920,7 +920,7 @@ main() {
export TEST_REPORT_DIR="$REPORT_DIR" export TEST_REPORT_DIR="$REPORT_DIR"
gha_group "Test: Behave regression tests" gha_group "Test: Behave regression tests"
if python -m behave \ if uv run --project ../../engine --locked --group cucumber python -m behave \
-f behave_html_formatter:HTMLFormatter -o "$CUCUMBER_REPORT" \ -f behave_html_formatter:HTMLFormatter -o "$CUCUMBER_REPORT" \
-f pretty \ -f pretty \
--junit --junit-directory "$CUCUMBER_JUNIT_DIR"; then --junit --junit-directory "$CUCUMBER_JUNIT_DIR"; then