mirror of
https://github.com/Stirling-Tools/Stirling-PDF.git
synced 2026-09-03 05:10:16 +03:00
# Description of Changes This PR modernizes the project's Python tooling across GitHub Actions by migrating CI workflows from pip-based dependency management to `uv` and aligning Python execution with the engine project's managed environment. ### What was changed - Replaced `actions/setup-python` and ad-hoc `pip install` steps with `astral-sh/setup-uv` across CI workflows. - Configured shared `uv` dependency caching using `engine/pyproject.toml` and `engine/uv.lock`. - Updated Python script execution to use `uv run --project engine --locked` for a consistent runtime environment. - Replaced package installation steps with `uv sync` for the required dependency groups (e.g. `tools` and `cucumber`). - Added Docker image build validation for both production and development AI engine images. - Updated workflow cache configuration and Docker build context where required. - Removed obsolete Python requirements files that are no longer needed after the migration. - Applied minor Python code modernizations, including import cleanup, modern built-in generic type annotations (`list[...]`, `tuple[...]`, `float | None`), and small style improvements. - Removed unnecessary Python formatter/linter extensions from the development container configuration. ### Why the change was made - Standardize Python dependency management across the repository. - Reduce duplicated dependency installation logic in CI. - Improve workflow performance through shared dependency caching. - Ensure all Python utilities execute against the same locked dependency set managed by the engine project. - Simplify long-term maintenance by eliminating legacy requirements files and pip-specific workflow steps. --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### Translations (if applicable) - [ ] I ran [`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have run `task check` to verify linters, typechecks, and tests pass - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing) for more details. --------- Signed-off-by: Carsten Drewes <c.drewes@stud.uni-hannover.de> Co-authored-by: albanobattistella <34811668+albanobattistella@users.noreply.github.com> Co-authored-by: kastenherri <116314318+kastenherri@users.noreply.github.com> Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: James Brunton <jbrunton96@gmail.com>
250 lines
10 KiB
YAML
250 lines
10 KiB
YAML
name: Backend build, format check, and coverage
|
||
|
||
# Reusable workflow called from build.yml. Runs the backend build matrix
|
||
# (JDK 25 × every flavor), Spotless formatting check, JUnit, and
|
||
# posts Jacoco coverage to PRs.
|
||
#
|
||
# Flavor axis (maps to STIRLING_FLAVOR in settings.gradle):
|
||
# core - DISABLE_ADDITIONAL_FEATURES=true, no proprietary, no saas
|
||
# proprietary - default build, no saas
|
||
# saas - proprietary + the saas subproject (build + JUnit only,
|
||
# never any runtime/integration testing)
|
||
on:
|
||
workflow_call:
|
||
|
||
permissions:
|
||
contents: read
|
||
actions: read
|
||
security-events: write
|
||
pull-requests: write
|
||
|
||
jobs:
|
||
build:
|
||
runs-on: ubuntu-latest
|
||
strategy:
|
||
fail-fast: false
|
||
matrix:
|
||
jdk-version: [25]
|
||
flavor: [core, proprietary, saas]
|
||
steps:
|
||
- name: Harden Runner
|
||
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
|
||
with:
|
||
egress-policy: audit
|
||
- name: Checkout repository
|
||
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
|
||
|
||
- name: Set up JDK ${{ matrix.jdk-version }}
|
||
uses: actions/setup-java@b6effb05e454b25005698d916606bdc6ffcbf961 # v5.7.0
|
||
with:
|
||
java-version: ${{ matrix.jdk-version }}
|
||
distribution: "temurin"
|
||
|
||
- name: Cache Gradle User Home
|
||
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
|
||
with:
|
||
path: |
|
||
~/.gradle/caches
|
||
~/.gradle/wrapper
|
||
key: gradle-${{ runner.os }}-${{ runner.arch }}-jdk-${{ matrix.jdk-version }}-${{ hashFiles('gradle/wrapper/gradle-wrapper.properties', 'gradle/libs.versions.toml', 'settings.gradle', 'build.gradle', 'app/**/build.gradle', 'gradle/**/*.gradle') }}
|
||
restore-keys: |
|
||
gradle-${{ runner.os }}-${{ runner.arch }}-jdk-${{ matrix.jdk-version }}-
|
||
gradle-${{ runner.os }}-${{ runner.arch }}-
|
||
|
||
- name: Install Task
|
||
uses: go-task/setup-task@01a4adf9db2d14c1de7a560f09170b6e0df736aa # v2.1.0
|
||
- name: Check Java formatting (Spotless)
|
||
# Runs once per matrix combination - pick the cheapest leg
|
||
# (core - no proprietary, no saas) so we don't wait for the
|
||
# heavier flavors just to fail formatting.
|
||
if: matrix.jdk-version == 25 && matrix.flavor == 'core'
|
||
id: spotless-check
|
||
run: task backend:format:check
|
||
continue-on-error: true
|
||
env:
|
||
MAVEN_USER: ${{ secrets.MAVEN_USER }}
|
||
MAVEN_PASSWORD: ${{ secrets.MAVEN_PASSWORD }}
|
||
MAVEN_PUBLIC_URL: ${{ secrets.MAVEN_PUBLIC_URL }}
|
||
|
||
- name: Comment on backend format check failure
|
||
# Only post a comment on PRs. github-script's PR helpers need an
|
||
# issue/PR number, which doesn't exist on merge_group runs.
|
||
if: steps.spotless-check.outcome == 'failure' && github.event_name == 'pull_request'
|
||
continue-on-error: true
|
||
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
|
||
with:
|
||
script: |
|
||
const marker = '<!-- java-formatting-check -->';
|
||
const body = [
|
||
marker,
|
||
'### Backend Format Check Failed',
|
||
'',
|
||
'There are formatting issues in your Java code that will need to be fixed before they can be merged in.',
|
||
'',
|
||
'Run `task backend:format` to auto-fix, then commit and push the changes.',
|
||
].join('\n');
|
||
const { data: comments } = await github.rest.issues.listComments({
|
||
owner: context.repo.owner,
|
||
repo: context.repo.repo,
|
||
issue_number: context.issue.number,
|
||
});
|
||
const existing = comments.find(c => c.body.includes(marker));
|
||
if (existing) {
|
||
await github.rest.issues.updateComment({
|
||
owner: context.repo.owner,
|
||
repo: context.repo.repo,
|
||
comment_id: existing.id,
|
||
body,
|
||
});
|
||
} else {
|
||
await github.rest.issues.createComment({
|
||
owner: context.repo.owner,
|
||
repo: context.repo.repo,
|
||
issue_number: context.issue.number,
|
||
body,
|
||
});
|
||
}
|
||
|
||
- name: Fail if backend format check failed
|
||
if: steps.spotless-check.outcome == 'failure'
|
||
run: |
|
||
echo "============================================"
|
||
echo " Backend Format Check Failed"
|
||
echo "============================================"
|
||
echo ""
|
||
echo "There are formatting issues in your Java code"
|
||
echo "that will need to be fixed before they can be"
|
||
echo "merged in."
|
||
echo ""
|
||
echo "Run 'task backend:format' to auto-fix, then"
|
||
echo "commit and push the changes."
|
||
echo "============================================"
|
||
exit 1
|
||
|
||
- name: Remove backend format check comment on success
|
||
if: steps.spotless-check.outcome == 'success' && github.event_name == 'pull_request'
|
||
continue-on-error: true
|
||
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
|
||
with:
|
||
script: |
|
||
const marker = '<!-- java-formatting-check -->';
|
||
const { data: comments } = await github.rest.issues.listComments({
|
||
owner: context.repo.owner,
|
||
repo: context.repo.repo,
|
||
issue_number: context.issue.number,
|
||
});
|
||
const existing = comments.find(c => c.body.includes(marker));
|
||
if (existing) {
|
||
await github.rest.issues.deleteComment({
|
||
owner: context.repo.owner,
|
||
repo: context.repo.repo,
|
||
comment_id: existing.id,
|
||
});
|
||
}
|
||
|
||
- name: Build with Gradle (flavor=${{ matrix.flavor }})
|
||
# STIRLING_FLAVOR is read by settings.gradle and expands into the
|
||
# right combination of DISABLE_ADDITIONAL_FEATURES + ENABLE_SAAS
|
||
# so we don't have to set them by hand. The saas flavor pulls in
|
||
# the app/saas subproject (unit tests only - no runtime tests).
|
||
run: task backend:build:ci
|
||
env:
|
||
MAVEN_USER: ${{ secrets.MAVEN_USER }}
|
||
MAVEN_PASSWORD: ${{ secrets.MAVEN_PASSWORD }}
|
||
MAVEN_PUBLIC_URL: ${{ secrets.MAVEN_PUBLIC_URL }}
|
||
STIRLING_FLAVOR: ${{ matrix.flavor }}
|
||
|
||
- name: Check Test Reports Exist
|
||
if: always()
|
||
run: |
|
||
# Common + core + proprietary always build (proprietary is
|
||
# excluded only at runtime, not from the gradle subproject
|
||
# graph). Saas builds add a fourth report dir.
|
||
declare -a dirs=(
|
||
"app/core/build/reports/tests/"
|
||
"app/core/build/test-results/"
|
||
"app/common/build/reports/tests/"
|
||
"app/common/build/test-results/"
|
||
"app/proprietary/build/reports/tests/"
|
||
"app/proprietary/build/test-results/"
|
||
)
|
||
if [ "${{ matrix.flavor }}" = "saas" ]; then
|
||
dirs+=("app/saas/build/reports/tests/" "app/saas/build/test-results/")
|
||
fi
|
||
for dir in "${dirs[@]}"; do
|
||
if [ ! -d "$dir" ]; then
|
||
echo "Missing $dir"
|
||
exit 1
|
||
fi
|
||
done
|
||
|
||
- name: Upload Test Reports
|
||
if: always()
|
||
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
|
||
with:
|
||
name: test-reports-jdk-${{ matrix.jdk-version }}-flavor-${{ matrix.flavor }}
|
||
path: |
|
||
app/**/build/reports/jacoco/test
|
||
app/**/build/reports/tests/
|
||
app/**/build/test-results/
|
||
app/**/build/reports/problems/
|
||
build/reports/problems/
|
||
retention-days: 3
|
||
if-no-files-found: warn
|
||
|
||
- name: Install uv
|
||
if: always() && matrix.flavor == 'saas'
|
||
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
|
||
with:
|
||
enable-cache: true
|
||
cache-dependency-glob: |
|
||
engine/pyproject.toml
|
||
engine/uv.lock
|
||
|
||
- name: JaCoCo coverage step summary
|
||
# Only the saas leg posts the JUnit summary - it's a strict
|
||
# superset of the core + proprietary legs (same .exec files plus
|
||
# the saas subproject). Posting from all three would mean three
|
||
# near-identical tables crowding out the aggregate report.
|
||
if: always() && matrix.flavor == 'saas'
|
||
run: |
|
||
uv run --project engine --locked --group tools python scripts/coverage-summary.py \
|
||
--title "Backend JUnit coverage (JDK ${{ matrix.jdk-version }})" \
|
||
--jacoco "common=app/common/build/reports/jacoco/test/jacocoTestReport.xml" \
|
||
--jacoco "core=app/core/build/reports/jacoco/test/jacocoTestReport.xml" \
|
||
--jacoco "proprietary=app/proprietary/build/reports/jacoco/test/jacocoTestReport.xml" \
|
||
--jacoco "saas=app/saas/build/reports/jacoco/test/jacocoTestReport.xml" \
|
||
--github-step-summary
|
||
|
||
- name: Upload raw JUnit .exec for aggregate merge
|
||
# Same dedup rationale as the summary step: upload from the saas
|
||
# leg only (the most complete set, includes app/saas/.../test.exec)
|
||
# so the aggregate workflow merges the union rather than three
|
||
# overlapping subsets.
|
||
#
|
||
# Separate artifact from the HTML reports so the aggregate
|
||
# workflow can grab just the .exec files with a name pattern
|
||
# (`jacoco-exec-*`) instead of unpacking the whole test-reports
|
||
# tarball.
|
||
if: always() && matrix.flavor == 'saas'
|
||
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
|
||
with:
|
||
name: jacoco-exec-junit-jdk-${{ matrix.jdk-version }}
|
||
path: app/*/build/jacoco/*.exec
|
||
retention-days: 7
|
||
if-no-files-found: warn
|
||
|
||
- name: Add coverage to PR (flavor=${{ matrix.flavor }}, JDK=${{ matrix.jdk-version }})
|
||
# The action only supports the pull_request event (it posts a PR comment),
|
||
# so skip it for merge_group runs and workflow_dispatch.
|
||
if: github.event_name == 'pull_request'
|
||
id: jacoco
|
||
uses: madrapps/jacoco-report@e51ce1f46f7f8b5331593f935e59cbaf44b84920 # v1.8.0
|
||
with:
|
||
paths: |
|
||
${{ github.workspace }}/**/build/reports/jacoco/test/jacocoTestReport.xml
|
||
token: ${{ secrets.GITHUB_TOKEN }}
|
||
min-coverage-overall: 10
|
||
min-coverage-changed-files: 0
|
||
comment-type: summary
|