Skip to content

Phase 14: Continuous Integration

Prerequisite: All prior phases (01-13) are implemented and tests pass. The only existing GitHub Actions workflow is docs.yml (MkDocs deploy on push to main). This phase adds CI for linting, testing, building, and coverage reporting.

1. Overview

The project has 800+ unit test functions across 46 files and ~80 browser test functions across 10 files (9 using Playwright). All LLM calls are mocked — no API keys are needed in CI. Git is used in some tests via tmp_path fixtures (safe for CI with global git config). The build system is uv with hatchling backend.

This phase creates a GitHub Actions CI workflow that runs on every push to main and every pull request. It adds coverage reporting via Codecov, and places status badges on the README.

What This Phase Does

  1. Adds .github/workflows/ci.yml with four jobs: Lint, Unit Tests (matrix), Browser Tests, Build
  2. Adds pytest-cov to dev dependencies and configures coverage in pyproject.toml
  3. Integrates Codecov for coverage reporting
  4. Adds CI status, coverage, Python version, license, and docs badges to README.md

What This Phase Does NOT Do

  • No changes to test code, engine, agents, or UI
  • No deployment or release automation (only CI)
  • No Docker or containerization

2. Architecture

2.1 Workflow Structure

ci.yml
  ├── lint          (ubuntu-latest, fast, no deps)
  │   ├── ruff check .
  │   └── ruff format --check .
  ├── test          (ubuntu-latest, Python 3.11 + 3.12 matrix)
  │   ├── uv sync --dev --extra web
  │   ├── pytest tests/ --ignore=tests/web/ --cov
  │   └── upload coverage to Codecov (3.12 only)
  ├── browser-test  (ubuntu-latest, Python 3.12 only)
  │   ├── uv sync --dev --extra web
  │   ├── playwright install chromium --with-deps
  │   └── pytest tests/web/
  └── build         (ubuntu-latest)
      └── uv build

2.2 Key Design Decisions

  1. astral-sh/setup-uv@v5 for uv installation — official action, handles caching.
  2. uvx ruff@0.11.4 for lint — isolated tool execution, no uv sync needed. Pins version to match prek.toml.
  3. Concurrency group cancels in-progress runs on the same PR to save CI minutes.
  4. Browser tests use continue-on-error: true — they spawn a live NiceGUI subprocess and can be flaky on shared runners. Remove once proven stable.
  5. Coverage uploaded only on Python 3.12 to avoid conflating matrix results.
  6. fail-fast: false in test matrix so coverage upload isn't cancelled by a 3.11 failure.
  7. Git identity configured for tests that use gitpython (test_git_save.py, test_save_versioning.py).
  8. Unit test job installs --extra web because 3 unit tests in tests/ (not tests/web/) import from theact.web.* which requires nicegui: test_web_commands.py, test_web_components.py, test_web_session.py.

3. Implementation Steps

Step 1: Add pytest-cov and configure coverage

File: pyproject.toml

  • Add "pytest-cov>=6.0" to [dependency-groups] dev
  • Add [tool.pytest.ini_options] with testpaths = ["tests"]
  • Add [tool.coverage.run] with source = ["theact"], omitting cli/ and web/ (presentation layers)
  • Add [tool.coverage.report] with standard exclusions
  • Run uv lock to update lockfile

Step 2: Create CI workflow

File: .github/workflows/ci.yml

Four jobs: lint, test (3.11 + 3.12 matrix), browser-test, build. See Section 2.1.

Step 3: Create Codecov configuration

File: codecov.yml (repo root)

  • Project target: auto with 2% threshold
  • Patch target: 80% for new/changed lines
  • Condensed PR comments, only when coverage changes

Step 4: Add README badges

File: README.md

Insert before # TheAct: CI, codecov, Python 3.11+, MIT license, docs badges.

Step 5: Update CLAUDE.md

Add CI section documenting the workflow and update the plans list.

4. Tests That Need CI Attention

  • No tests need to be disabled. All LLM calls are mocked. No @pytest.mark.skip decorators exist.
  • Git identity required: test_git_save.py and test_save_versioning.py use gitpython with tmp_path — need git config --global user.email/name in CI.
  • test_safety.py has skipif not _HAS_FILELOCK — runs in browser test job (has --extra web), auto-skips in unit test job. Works correctly as-is.
  • Browser tests spawn a NiceGUI subprocess on port 8090. This works in CI but can be flaky due to timing — hence continue-on-error.

5. Verification

Local

uvx ruff@0.11.4 check . && uvx ruff@0.11.4 format --check .
uv run pytest tests/ --ignore=tests/web/ --cov --cov-report=term -v
uv run pytest tests/web/ -v
uv build && ls dist/*.whl

After Push

  1. All four CI jobs run at https://github.com/reubenjohn/theact/actions
  2. Lint job completes in under 30 seconds
  3. Test job passes on both Python 3.11 and 3.12
  4. Coverage appears on Codecov
  5. Browser Tests job runs (may be yellow due to continue-on-error)
  6. Build job produces a wheel
  7. README badges render correctly

6. Manual Setup Required

  • Codecov token: Add CODECOV_TOKEN as a GitHub repository secret (get from codecov.io after linking the repo)

7. Dependencies

Add to [dependency-groups] dev in pyproject.toml: 

"pytest-cov>=6.0",