CI: FT stages run in parallel, --parallel dropped intra-stage — bud-btn click sites bypass scroll-into-view ; .woodpecker/main.yaml restructures the FT split — test-FTs-non-room + test-FTs-room now both depends_on: test-two-browser-FTs (instead of room serially depending on non-room) so they fan out + run concurrently, each w. its own DATABASE_URL: sqlite:////tmp/test_db_{non_room,room}.sqlite3 outside the shared workspace mount so the two stages can't see each other's SQLite file + the #296 EOFError-on-half-created-test-db blocker is gone; --parallel flag dropped from both manage.py test invocations because the empirical wall-clock from #302-304 (151 tests in ~42 min, 83 tests in ~16 min — ~16-17s/test avg either way) shows ~1-1.5x speedup at best — Firefox spawn cost (~3-5s cold-start × 38 tests/worker) + RAM pressure (4 headless Firefoxes ≈ 1.5-2GB working set on a quadcore DO droplet) + SQLite file-lock contention eat most of the gain the cores would otherwise give, while the contention amplifies every transient-DOM flake we've spent the last 2 days chasing (login-race in #300/303 → 054b0aa + ad0041d, gecko-perms in #302/303 → ad0041d, ElementNotInteractable in #304 → this commit, Jasmine-timeout in #303 → ad0041d); stage-level parallelism gets back the wall-clock reduction (~16min savings vs serial stages) w.o. amplifying within-stage contention — net wall clock should be ~60-65min for both FT stages running concurrently vs ~58min today, but flake exposure drops dramatically; downstream screendumps + build-and-push already list both FT steps in their depends_on so they naturally gate on the slower of the two ; test_core_bud_btn.py wraps 4 unwrapped find_element(...).click() sites in wait_for(execute_script("arguments[0].click()", btn)) — the _open_panel_and_invite helper feeding 6 GatekeeperBudBtnAsyncInviteTest tests + the standalone GatekeeperBudBtnDuplicateInviteErrorTest test_duplicate_invite_shows_error_brief_and_fyi_flashes_slot click + both .note-banner--duplicate .note-banner__fyi clicks (one on the share-flow at line 433, one on the gatekeeper-invite-flow at line 622) — pipeline #304 errored on the OK button click w. ElementNotInteractableException: Element <button id="id_bud_ok"> could not be scrolled into view because the post-send_keys autocomplete dropdown on _bud_invite_panel.html briefly overlapped the OK button under CI contention so Firefox refused the scroll-into-view; same pattern + same fix shape as confirm_guard in base.py — execute_script bypasses Selenium's scroll-into-view gate entirely + wait_for absorbs any leftover transient state via WebDriverException retry ; commit only ships the test-side fix + CI restructure; if the CI changes work as intended (green pipeline #305) the docker-rebuild option for python-tdd-ci:latest still stands as the next ~5min/pipeline win, separately filed in project_ci_remove_pip_install_deferred.md — TDD
Code architected by Disco DeDisco <discodedisco@outlook.com> Git commit message Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -64,31 +64,29 @@ steps:
|
||||
- "requirements.txt"
|
||||
- ".woodpecker/main.yaml"
|
||||
|
||||
# ── FT split (sequential for now) ─────────────────────────────────────
|
||||
# ── FT split (stage-parallel, intra-stage sequential) ─────────────────
|
||||
#
|
||||
# test_game_room_* is the heaviest cluster — 9 Selenium-driven room-flow
|
||||
# FTs that historically dominate the FT step wall-clock (~70% of the
|
||||
# ~40-min single-step runs). Split off into its own step (`test-FTs-room`)
|
||||
# so the partition is visible in the pipeline view; the non-room bucket
|
||||
# runs first as `test-FTs-non-room`. test-FTs-non-room depends on
|
||||
# test-two-browser-FTs (collectstatic'd assets in shared workspace);
|
||||
# test-FTs-room depends on test-FTs-non-room.
|
||||
# is `test-FTs-non-room`. Both depend on test-two-browser-FTs only, so
|
||||
# they fan out + run concurrently.
|
||||
#
|
||||
# NOTE — currently sequential, not parallel. Both FT steps share the
|
||||
# workspace AND fall back to SQLite (only test-UTs-n-ITs has
|
||||
# `DATABASE_URL` pointing at the postgres service). When they ran
|
||||
# concurrently (both depending on test-two-browser-FTs), the second
|
||||
# step to start hit a half-created `src/test_db.sqlite3` and prompted
|
||||
# "Type 'yes' to delete the existing test database" → EOFError under
|
||||
# non-interactive CI stdin. See pipeline run #296.
|
||||
# The previous SQLite-collision blocker (pipeline #296: second step
|
||||
# started against the first step's half-created `src/test_db.sqlite3`
|
||||
# → Django interactive prompt → EOFError under non-interactive CI
|
||||
# stdin) is resolved by giving each step a distinct `DATABASE_URL`
|
||||
# pointing at its own sqlite file under /tmp — outside the shared
|
||||
# workspace mount so the two stages can't see each other's DB.
|
||||
#
|
||||
# To re-parallelise later, give each step its own DB:
|
||||
# - point `DATABASE_URL` at distinct sqlite paths per step
|
||||
# (e.g. `sqlite:////tmp/test_db_room.sqlite3` vs `_non_room`), or
|
||||
# - have each step point at its own postgres DB (the existing
|
||||
# `postgres` service already exposes one; spin up a second or
|
||||
# `CREATE DATABASE` per step).
|
||||
# Until then, test-FTs-room runs after test-FTs-non-room.
|
||||
# `--parallel` is dropped from both steps. Empirically (pipelines
|
||||
# #302-304) it was giving ~1-1.5x speedup at most on these Selenium
|
||||
# FTs — Firefox spawn cost + RAM pressure + SQLite file-lock contention
|
||||
# eat most of the gain — while amplifying every transient-DOM flake
|
||||
# (login-race, gecko-perms, ElementNotInteractable, Jasmine-timeout).
|
||||
# Stage-level parallelism gives the same wall-clock reduction without
|
||||
# contention amplification.
|
||||
|
||||
- name: test-FTs-non-room
|
||||
image: gitea.earthmanrpg.me/discoman/python-tdd-ci:latest
|
||||
@@ -96,6 +94,9 @@ steps:
|
||||
- test-two-browser-FTs
|
||||
environment:
|
||||
HEADLESS: 1
|
||||
# /tmp path (not workspace-relative) so the parallel test-FTs-room
|
||||
# step can't see this DB + vice versa. See split-rationale above.
|
||||
DATABASE_URL: sqlite:////tmp/test_db_non_room.sqlite3
|
||||
CELERY_BROKER_URL: redis://redis:6379/0
|
||||
REDIS_URL: redis://redis:6379/1
|
||||
STRIPE_SECRET_KEY:
|
||||
@@ -109,7 +110,7 @@ steps:
|
||||
# Every FT file EXCEPT test_game_room_* — that cluster runs in
|
||||
# test-FTs-room. Channels + two-browser tags already covered upstream.
|
||||
# `ls | grep -v | sed` enumerates module dotted-paths from filenames.
|
||||
- python manage.py test --parallel --exclude-tag=channels --exclude-tag=two-browser $(ls functional_tests/test_*.py | grep -v 'test_game_room_' | sed 's|/|.|g;s|\.py||')
|
||||
- python manage.py test --exclude-tag=channels --exclude-tag=two-browser $(ls functional_tests/test_*.py | grep -v 'test_game_room_' | sed 's|/|.|g;s|\.py||')
|
||||
when:
|
||||
- event: push
|
||||
path:
|
||||
@@ -120,9 +121,12 @@ steps:
|
||||
- name: test-FTs-room
|
||||
image: gitea.earthmanrpg.me/discoman/python-tdd-ci:latest
|
||||
depends_on:
|
||||
- test-FTs-non-room
|
||||
- test-two-browser-FTs
|
||||
environment:
|
||||
HEADLESS: 1
|
||||
# /tmp path (not workspace-relative) so test-FTs-non-room can't see
|
||||
# this DB + vice versa. See split-rationale above.
|
||||
DATABASE_URL: sqlite:////tmp/test_db_room.sqlite3
|
||||
CELERY_BROKER_URL: redis://redis:6379/0
|
||||
REDIS_URL: redis://redis:6379/1
|
||||
STRIPE_SECRET_KEY:
|
||||
@@ -135,9 +139,9 @@ steps:
|
||||
- cd ./src
|
||||
# Heavy Selenium room flows — 9 files (deck_contrib, gatekeeper,
|
||||
# invite, select_role/sea/sig/sky, tray, tray_tooltip) isolated
|
||||
# into their own sub-step. Runs after test-FTs-non-room (shared
|
||||
# SQLite test DB; see split-rationale comment above).
|
||||
- python manage.py test --parallel --exclude-tag=channels --exclude-tag=two-browser $(ls functional_tests/test_game_room_*.py | sed 's|/|.|g;s|\.py||')
|
||||
# into their own sub-step. Runs in parallel w. test-FTs-non-room
|
||||
# (distinct DATABASE_URL paths under /tmp; see split-rationale).
|
||||
- python manage.py test --exclude-tag=channels --exclude-tag=two-browser $(ls functional_tests/test_game_room_*.py | sed 's|/|.|g;s|\.py||')
|
||||
when:
|
||||
- event: push
|
||||
path:
|
||||
|
||||
@@ -427,9 +427,12 @@ class BudBtnDuplicateShareErrorTest(FunctionalTest):
|
||||
self.assertNotIn("bud-duplicate-flash", chip.get_attribute("class") or "")
|
||||
|
||||
# FYI dismisses + applies flash class
|
||||
self.browser.find_element(
|
||||
By.CSS_SELECTOR, ".note-banner--duplicate .note-banner__fyi"
|
||||
).click()
|
||||
self.wait_for(lambda: self.browser.execute_script(
|
||||
"arguments[0].click()",
|
||||
self.browser.find_element(
|
||||
By.CSS_SELECTOR, ".note-banner--duplicate .note-banner__fyi"
|
||||
),
|
||||
))
|
||||
self.wait_for(lambda: self.assertEqual(
|
||||
self.browser.find_elements(By.CSS_SELECTOR, ".note-banner--duplicate"),
|
||||
[],
|
||||
@@ -503,9 +506,17 @@ class GatekeeperBudBtnAsyncInviteTest(FunctionalTest):
|
||||
lambda: self.browser.find_element(By.ID, "id_recipient")
|
||||
)
|
||||
recipient_input.send_keys(recipient)
|
||||
self.browser.find_element(
|
||||
By.CSS_SELECTOR, "#id_bud_panel .btn.btn-confirm"
|
||||
).click()
|
||||
# wait_for + execute_script bypasses Selenium's scroll-into-view
|
||||
# gate — under CI contention the post-send_keys autocomplete dropdown
|
||||
# can briefly overlap the OK button + Firefox refuses the click w.
|
||||
# ElementNotInteractableException (pipeline #304). Same shape as
|
||||
# confirm_guard in base.py.
|
||||
self.wait_for(lambda: self.browser.execute_script(
|
||||
"arguments[0].click()",
|
||||
self.browser.find_element(
|
||||
By.CSS_SELECTOR, "#id_bud_panel .btn.btn-confirm"
|
||||
),
|
||||
))
|
||||
return bud_btn
|
||||
|
||||
def test_invite_creates_room_invite(self):
|
||||
@@ -579,7 +590,13 @@ class GatekeeperBudBtnDuplicateInviteErrorTest(FunctionalTest):
|
||||
btn.click()
|
||||
recipient = self.wait_for(lambda: self.browser.find_element(By.ID, "id_recipient"))
|
||||
recipient.send_keys("alice@test.io")
|
||||
self.browser.find_element(By.CSS_SELECTOR, "#id_bud_panel .btn.btn-confirm").click()
|
||||
# See _open_panel_and_invite (above) for the bypass rationale.
|
||||
self.wait_for(lambda: self.browser.execute_script(
|
||||
"arguments[0].click()",
|
||||
self.browser.find_element(
|
||||
By.CSS_SELECTOR, "#id_bud_panel .btn.btn-confirm"
|
||||
),
|
||||
))
|
||||
|
||||
title = self.wait_for(lambda: self.browser.find_element(
|
||||
By.CSS_SELECTOR, ".note-banner--duplicate .note-banner__title"
|
||||
@@ -600,9 +617,12 @@ class GatekeeperBudBtnDuplicateInviteErrorTest(FunctionalTest):
|
||||
)
|
||||
self.assertNotIn("bud-duplicate-flash", slot.get_attribute("class") or "")
|
||||
|
||||
self.browser.find_element(
|
||||
By.CSS_SELECTOR, ".note-banner--duplicate .note-banner__fyi"
|
||||
).click()
|
||||
self.wait_for(lambda: self.browser.execute_script(
|
||||
"arguments[0].click()",
|
||||
self.browser.find_element(
|
||||
By.CSS_SELECTOR, ".note-banner--duplicate .note-banner__fyi"
|
||||
),
|
||||
))
|
||||
self.wait_for(lambda: self.assertEqual(
|
||||
self.browser.find_elements(By.CSS_SELECTOR, ".note-banner--duplicate"),
|
||||
[],
|
||||
|
||||
Reference in New Issue
Block a user