created CD pipeline to complement the existing gitea/woodpecker CI pipeline: build, push to registry, deploy to staging on main

2026-02-13 15:27:59 -05:00
parent e02385118f
commit 510874b87c
7 changed files with 322 additions and 21 deletions
--- a/.woodpecker.yaml
+++ b/.woodpecker.yaml
@@ -21,4 +21,32 @@ steps:
      - status: failure
    commands:
      - cat ./src/functional_tests/screendumps/*.html || echo "No screendumps found"
+
+  - name: build-and-push
+    image: docker:cli
+    environment:
+      REGISTRY_PASSWORD:
+        from_secret: gitea_registry_password
+    commands: 
+      - echo "$REGISTRY_PASSWORD" | docker login gitea.earthmanrpg.me -u discoman --password-stdin
+      - docker build -t gitea.earthmanrpg.me/discoman/gamearray:latest .
+      - docker push gitea.earthmanrpg.me/discoman/gamearray:latest
+    when:
+      - branch: main
+      - event: push
+
+  - name: deploy
+    image: alpine
+    environment:
+      SSH_KEY:
+        from_secret: deploy_ssh_key
+    commands:
+      - apk add --no-cache openssh-client
+      - mkdir -p ~/.ssh
+      - printf '%s\n' "$SSH_KEY" > ~/.ssh/id_ed25519
+      - chmod 600 ~/.ssh/id_ed25519
+      - ssh -o StrictHostKeyChecking=no discoman@staging.earthmanrpg.me /opt/gamearray/deploy.sh
+    when:
+      - branch: main
+      - event: push
    
--- a/infra/CD-PLAN.md
+++ b/infra/CD-PLAN.md
@@ -0,0 +1,209 @@
+# Continuous Deployment: Auto-deploy to staging when CI passes
+
+## Context
+
+CI is complete (Percival ch. 25). The Woodpecker pipeline runs 57 UTs + 9 FTs on every push to Gitea, but deployment to staging is still manual (`ansible-playbook infra/deploy-playbook.yaml`). This plan adds CD: when tests pass on the `main` branch, the pipeline automatically builds the Docker image, pushes it to the Gitea container registry, and deploys it to `staging.earthmanrpg.me`.
+
+**Architecture:** Build in CI pipeline -> push to Gitea registry -> SSH to staging -> pull + restart
+
+---
+
+## Stage 1: Enable Docker socket in Woodpecker pipeline steps
+
+The Woodpecker agent already has the Docker socket mounted (line 53 of `infra/cicd/docker-compose.yaml`), but pipeline steps (containers the agent launches) don't inherit it. We need to tell the agent to pass it through.
+
+### 1a. Edit `infra/cicd/docker-compose.yaml` -- add env var to `woodpecker-agent`
+
+Add this to the agent's `environment:` block:
+
+```yaml
+- WOODPECKER_BACKEND_DOCKER_VOLUMES=/var/run/docker.sock:/var/run/docker.sock
+```
+
+**Why:** This makes the host Docker socket available inside every pipeline step container. Without it, the `docker build` and `docker push` commands in our pipeline step would fail with "cannot connect to Docker daemon."
+
+### 1b. Re-deploy CI stack
+
+From WSL:
+
+```bash
+ansible-playbook infra/cicd-playbook.yaml -i infra/inventory.ini --ask-vault-pass
+```
+
+This uploads the updated docker-compose and restarts the CI services.
+
+---
+
+## Stage 2: Prepare staging server for CD
+
+Currently `deploy-playbook.yaml` does everything in one shot: installs Docker, builds the image locally, uploads it, runs it with inline `env:` vars, copies static files, migrates. For CD, we need to split "one-time setup" from "per-deploy actions."
+
+### 2a. Create `infra/gamearray.env.j2` -- container env file template (new file)
+
+```
+DJANGO_DEBUG_FALSE=1
+DJANGO_SECRET_KEY={{ secret_key.content | b64decode }}
+DJANGO_ALLOWED_HOST={{ django_allowed_host }}
+DJANGO_DB_PATH=/home/nonroot/db.sqlite3
+EMAIL_HOST_USER={{ email_host_user }}
+EMAIL_HOST_PASSWORD={{ email_host_password }}
+MAILGUN_API_KEY={{ mailgun_api_key }}
+```
+
+**Why:** Instead of passing env vars inline in `docker run`, we template a `.env` file onto the staging server. The deploy script references it with `--env-file`. This keeps secrets out of the pipeline YAML and on the server, managed by Ansible vault.
+
+### 2b. Create `infra/deploy.sh.j2` -- deploy script template (new file)
+
+```bash
+#!/bin/bash
+set -euo pipefail
+
+IMAGE=gitea.earthmanrpg.me/discoman/gamearray:latest
+
+echo "==> Pulling latest image..."
+docker pull "$IMAGE"
+
+echo "==> Stopping old container..."
+docker stop gamearray 2>/dev/null || true
+docker rm gamearray 2>/dev/null || true
+
+echo "==> Starting new container..."
+docker run -d --name gamearray \
+  --env-file /opt/gamearray/gamearray.env \
+  -p 127.0.0.1:8888:8888 \
+  "$IMAGE"
+
+echo "==> Running migrations..."
+docker exec gamearray ./manage.py migrate
+
+echo "==> Copying static files..."
+sudo docker cp gamearray:/src/static/. /var/www/gamearray/static/
+
+echo "==> Deploy complete."
+```
+
+**Why:** A single script the pipeline calls via SSH. Keeps the pipeline YAML clean and lets us test deploys manually too (`ssh staging /opt/gamearray/deploy.sh`).
+
+### 2c. Update `infra/deploy-playbook.yaml` -- add env file, deploy script, and registry login
+
+Add these tasks (after the existing "Read secret key" task):
+
+1. **Create `/opt/gamearray/` directory** on staging
+2. **Template `gamearray.env.j2`** to `/opt/gamearray/gamearray.env` (mode 0600)
+3. **Template `deploy.sh.j2`** to `/opt/gamearray/deploy.sh` (mode 0755)
+4. **Docker login to Gitea registry** -- so the deploy script can `docker pull`
+
+For the Docker login, use a Gitea access token (created in the browser, stored in the staging vault):
+
+```yaml
+- name: Login to Gitea container registry
+  ansible.builtin.command:
+    cmd: docker login gitea.earthmanrpg.me -u discoman -p {{ gitea_registry_token }}
+  no_log: true
+```
+
+**Vault additions** needed for staging (`infra/group_vars/staging/vault.yaml`):
+- `gitea_registry_token` -- a Gitea personal access token with `package:read` scope
+
+### 2d. Run the deploy playbook
+
+```bash
+ansible-playbook infra/deploy-playbook.yaml -i infra/inventory.ini -l staging --ask-vault-pass
+```
+
+This sets up the staging server: env file, deploy script, and registry auth.
+
+---
+
+## Stage 3: Woodpecker secrets
+
+Two secrets need to be added via the Woodpecker UI (`https://ci.earthmanrpg.me` -> repo settings -> Secrets):
+
+### 3a. `deploy_ssh_key`
+
+Generate a dedicated SSH key pair for CD:
+
+```bash
+ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519_cd_deploy -C "woodpecker-cd"
+```
+
+- Add the **public key** to `~/.ssh/authorized_keys` on staging (as `discoman` user)
+- Paste the **private key** contents into a Woodpecker secret named `deploy_ssh_key`
+
+### 3b. `gitea_registry_password`
+
+The Gitea access token (same one from Stage 2c, or a separate one with `package:read,package:write` scope).
+
+- Paste into a Woodpecker secret named `gitea_registry_password`
+- The `build-and-push` step will use this to `docker login` + `docker push`
+
+---
+
+## Stage 4: Update `.woodpecker.yaml`
+
+Add two new steps after `screendumps`:
+
+```yaml
+  - name: build-and-push
+    image: docker:cli
+    environment:
+      REGISTRY_PASSWORD:
+        from_secret: gitea_registry_password
+    commands:
+      - echo "$REGISTRY_PASSWORD" | docker login gitea.earthmanrpg.me -u discoman --password-stdin
+      - docker build -t gitea.earthmanrpg.me/discoman/gamearray:latest .
+      - docker push gitea.earthmanrpg.me/discoman/gamearray:latest
+    when:
+      - branch: main
+      - event: push
+
+  - name: deploy
+    image: alpine
+    environment:
+      SSH_KEY:
+        from_secret: deploy_ssh_key
+    commands:
+      - apk add --no-cache openssh-client
+      - mkdir -p ~/.ssh
+      - printf '%s\n' "$SSH_KEY" > ~/.ssh/id_ed25519
+      - chmod 600 ~/.ssh/id_ed25519
+      - ssh -o StrictHostKeyChecking=no discoman@staging.earthmanrpg.me /opt/gamearray/deploy.sh
+    when:
+      - branch: main
+      - event: push
+```
+
+**Why `docker:cli` instead of a Docker plugin?** Using plain Docker CLI with the socket pass-through is more transparent than a third-party plugin. You can see exactly what's happening.
+
+**Why `alpine` + manual SSH instead of `appleboy/drone-ssh`?** Same reason -- fewer moving parts, easier to debug. The `printf` + `ssh` pattern is dead simple.
+
+**Branch gating:** Both steps only run on pushes to `main`. Feature branch pushes still run tests but don't deploy.
+
+---
+
+## Stage 5: Test end-to-end
+
+1. Push a small visible change (e.g., tweak a template's page title) to `main` on Gitea
+2. Watch the Woodpecker pipeline at `https://ci.earthmanrpg.me`:
+   - `test-UTs` passes
+   - `test-FTs` passes
+   - `build-and-push` builds the image and pushes to the registry
+   - `deploy` SSHes to staging and runs the deploy script
+3. Visit `https://staging.earthmanrpg.me` and verify the change is live
+
+---
+
+## Files modified
+
+| File | Action |
+|------|--------|
+| `infra/cicd/docker-compose.yaml` | Add `WOODPECKER_BACKEND_DOCKER_VOLUMES` to agent |
+| `infra/gamearray.env.j2` | **New** -- container env file template |
+| `infra/deploy.sh.j2` | **New** -- deploy script template |
+| `infra/deploy-playbook.yaml` | Add tasks for env file, deploy script, registry login |
+| `infra/group_vars/staging/vault.yaml` | Add `gitea_registry_token` |
+| `.woodpecker.yaml` | Add `build-and-push` and `deploy` steps |
+
+## Known limitation
+
+**Ephemeral database:** The SQLite database still lives inside the container (deferred issue from memory). Every deploy wipes it. This is fine for staging right now but will need a volume mount or PostgreSQL before production CD.
--- a/infra/cicd/docker-compose.yaml
+++ b/infra/cicd/docker-compose.yaml
@@ -49,6 +49,7 @@ services:
      - WOODPECKER_SERVER=woodpecker-server:9000
      - WOODPECKER_AGENT_SECRET=${WOODPECKER_AGENT_SECRET}
      - WOODPECKER_MAX_WORKFLOWS=2
+      - WOODPECKER_BACKEND_DOCKER_VOLUMES=/var/run/docker.sock:/var/run/docker.sock
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    networks:
--- a/infra/deploy-playbook.yaml
+++ b/infra/deploy-playbook.yaml
@@ -94,6 +94,31 @@
        src: ~/.secret-key
      register: secret_key

+    - name: Create /opt/gamearray/ directory
+      ansible.builtin.file:
+        path: /opt/gamearray
+        state: directory
+      become: true
+
+    - name: Template gamearray.env to server
+      ansible.builtin.template:
+        src: gamearray.env.j2
+        dest: /opt/gamearray/gamearray.env
+        mode: "0600"
+      become: true
+
+    - name: Template deploy script to server
+      ansible.builtin.template:
+        src: deploy.sh.j2
+        dest: /opt/gamearray/deploy.sh
+        mode: "0755"
+      become: true
+
+    - name: Login to Gitea container registry
+      ansible.builtin.command:
+        cmd: docker login gitea.earthmanrpg.me -u discoman -p {{ gitea_registry_token }}
+      no_log: true
+
    - name: Ensure db.sqlite3 file exists outside container
      ansible.builtin.file:
        path: "{{ ansible_env.HOME }}/db.sqlite3"
--- a/infra/deploy.sh.j2
+++ b/infra/deploy.sh.j2
@@ -0,0 +1,25 @@
+#!/bin/bash
+set -euo pipefail
+
+IMAGE=gitea.earthmanrpg.me/discoman/gamearray:latest
+
+echo "==> Pulling latest image..."
+docker pull "$IMAGE"
+
+echo "==> Stopping old container..."
+docker stop gamearray 2>/dev/null || true
+docker rm gamearray 2>/dev/null || true
+
+echo "==> Starting new container..."
+docker run -d --name gamearray \
+    --env-file /opt/gamearray/gamearray.env \
+    -p 127.0.0.1:8888:8888 \
+    "$IMAGE"
+
+echo "==> Running migrations..."
+docker exec gamearray ./manage.py migrate
+
+echo "==> Copying static files..."
+sudo docker cp gamearray:/src/static/. /var/www/gamearray/static/
+
+echo "==> Deploy complete."
--- a/infra/gamearray.env.j2
+++ b/infra/gamearray.env.j2
@@ -0,0 +1,7 @@
+DJANGO_DEBUG_FALSE=1
+DJANGO_SECRET_KEY={{ secret_key.content | b64decode }}
+DJANGO_ALLOWED_HOST={{ django_allowed_host }}
+DJANGO_DB_PATH=/home/nonroot/db.sqlite3
+EMAIL_HOST_USER={{ email_host_user }}
+EMAIL_HOST_PASSWORD={{ email_host_password }}
+MAILGUN_API_KEY={{ mailgun_api_key }}
--- a/infra/group_vars/staging/vault.yaml
+++ b/infra/group_vars/staging/vault.yaml
@@ -1,22 +1,28 @@
 $ANSIBLE_VAULT;1.1;AES256
-33333861646565323863616363316265346132643135656533613763336462656361353562396163
-6461313530323231383264373737323934346239316232340a356130646163636436663162386438
-30636530373663643166616665313738303437636562643035303138613861656165663939613433
-3465316130343732660a376536373136636162316665386536363536306130383033623735396634
-66383262643630356536633366373933313637646539643137356330316463356139613738386661
-34663231656232666630653162653732363431613461396464623133303965636432626536306634
-64373530313532353531633263383335653239386530343330326237343862386436633666646235
-31613530383834656462366535643030356562313237363735386337356165663564336564623862
-34623863623738653735393734336635616135383036306231623464653432616265626233306230
-63646466323135363466393832636466646434303564653032323366346430306336363435653761
-30336437373231376261326264616131653833616236623365393334303834626162343761623037
-36666665663866643263383835626336353030626337303461396665343731666465653662396164
-34306261356130363037643637303632663830383331346334313336663163303730306265393031
-39653231373139616465326561313633306433653461653931663164363565363636316433323933
-61333536323936306538343336663966633161633565666231393261643062636239323264623364
-38376266393937376133366561663931356236396131376137653536636539613738356466363334
-32326165316434636631613366376235633337356135333531623861343039346261656239613036
-65303836323538373832646531343234666330363161343337623539633464303161343765363331
-35386233303563346662633239346363373931333764383233623161313965623266656364383037
-32393738356532346665613031346338363738666265303765363438663062663237353033393262
-6137
+36653566363731653435616430626663303038623766663561363231333163336165623863613964
+6164383861643530366438623465613565373032396331350a666163636431636663353162383531
+34306534656430653533303530613764336438616536343534663236333665323837636337333334
+3432643436636265610a313465396435616263386631353336326464333930613865313934313032
+38353362623937643234333466323063336535623666613366633263623034616638653566666463
+66323032653034376663623933306162313832643038653764643864666433376236643163663637
+63626334393963343934666665373764393066383866616461333063633664363436613031663036
+61343939343633393138666637646137376537393335663032383839306365613764303833323338
+33343936333730373362393466373238636666343762373134633962383237623335373634656330
+37363039393261313034306166656563333461353034646234323462623631393338383461363961
+33356564633637333630663464613265666264393435363238383530333861636365616362316130
+38353464343064616463636535316339336430323866303161393065363830356431386430666534
+61353961666333313536616661636631643630373337633262653662393863336264636431366634
+32323533383963393435343935616135663262633634356631363632396233383839326365396333
+64333232626465643438313132323661386235313063303036303631376537353666313532323766
+63633834336631633364333334373461333836666630353363343365323033653234356536643939
+30316538663230653636316532393931333936613733336366326239633362353666636436636136
+64656134663733376630316536616138613234383838316138616433353531396363316462626133
+62383431396465333634623066333565643332613935653532613536646632346533383362393330
+38646562393762346434663666313431363037636463306435663263386336343461303839346365
+63326432643662353830383736613636643866363765366132653563363036316265646531623433
+30303131323165653564333331353233373731333539346163613564343331373931633365633631
+66396332653436376430626564316639623362383635633134343234626462333162336464656438
+65393062333631373836303662326436333265373033353339356334633666363065636164343239
+64623535663633653130643764656539643339633061646437643366376261383137613439323934
+37343338336130313339356531333038613334393736353365366662313262653737623533616366
+346430623266646464353639386266313339