External/card-drop
Watch
5
0
Fork
You've already forked card-drop
 0
Code Issues 57 Pull requests 1 Projects 1 Releases 37 Packages Wiki Activity Actions

Split deployment workflows into 3 different stages #466

New issue
Closed
opened 2025-07-10 11:24:44 +01:00 by Vylpes · 2 comments
Vylpes commented 2025-07-10 11:24:44 +01:00
Owner
Copy link

Story points: 5

User story

As an operator, I want deploy workflows split into build → copy → deploy so artifacts are versioned in Forgejo Actions and runtime hosts only receive what the app needs to run.

Current behaviour

  • stage.yml and production.yml each have a build job that installs/builds/tests/lints then rsyncs the entire checkout to the host, then a deploy job that SSH-restarts MySQL (docker compose) + migrations + pm2.
  • No Actions artifact upload; copy is fused into the build job.
  • Stage deploy job incorrectly uses Forgejo environment: prod.

Acceptance criteria

  • GIVEN a stage or production deploy workflow runs
    WHEN the pipeline executes
    THEN it has three sequential jobs: build → copy → deploy.

  • GIVEN the build job
    WHEN install/build/test/lint succeed
    THEN a lean runtime package is uploaded as a Forgejo Actions artifact (only files required to run: e.g. dist/, package.json, yarn.lock, docker-compose.yml, and production node_modules/ — not source/tests/dev-only tree).

  • GIVEN the copy job
    WHEN it runs after build
    THEN it downloads that artifact and rsyncs it to the existing STAGE_RSYNC_PATH / PROD_RSYNC_PATH (same SSH/rsync secrets as today).

  • GIVEN the deploy job
    WHEN it runs after copy
    THEN runtime behaviour matches today: SSH → docker compose down/up → yarn db:up → pm2 restart (card-drop_stage / card-drop_prod), with the same env injection.

  • GIVEN the stage workflow
    WHEN the deploy job is configured
    THEN it uses Forgejo environment: stage (not prod).

  • GIVEN triggers
    WHEN unchanged branch/tag rules fire
    THEN stage still deploys from develop/hotfix/*/release/*, production from tags.

Subtasks

  • Refactor .forgejo/workflows/stage.yml into build / copy / deploy jobs
  • Refactor .forgejo/workflows/production.yml the same way
  • Upload lean Actions artifact in build; download + rsync in copy
  • Fix stage deploy environment: prod → stage
  • Verify deploy script still works against lean tree (db:up, pm2 start dist/bot.js)

Notes

  • Runtime stays host pm2 + compose MySQL (no Dockerize bot / Komodo).
  • Prefer shipping production node_modules from the runner (native deps like canvas) rather than installing on the host, unless we decide otherwise during implementation.
  • rsync --delete against a lean tree will remove old full-checkout files under the rsync root; ensure no required server-only files live there (or add excludes in a follow-up).
  • Out of scope: rsync exclude hardening, Komodo, app container image.
**Story points:** 5 ## User story As an operator, I want deploy workflows split into **build → copy → deploy** so artifacts are versioned in Forgejo Actions and runtime hosts only receive what the app needs to run. ## Current behaviour - `stage.yml` and `production.yml` each have a `build` job that installs/builds/tests/lints then **rsyncs the entire checkout** to the host, then a `deploy` job that SSH-restarts MySQL (`docker compose`) + migrations + `pm2`. - No Actions artifact upload; copy is fused into the build job. - Stage deploy job incorrectly uses Forgejo `environment: prod`. ## Acceptance criteria - **GIVEN** a stage or production deploy workflow runs **WHEN** the pipeline executes **THEN** it has three sequential jobs: **build** → **copy** → **deploy**. - **GIVEN** the **build** job **WHEN** install/build/test/lint succeed **THEN** a **lean** runtime package is uploaded as a **Forgejo Actions artifact** (only files required to run: e.g. `dist/`, `package.json`, `yarn.lock`, `docker-compose.yml`, and production `node_modules/` — not source/tests/dev-only tree). - **GIVEN** the **copy** job **WHEN** it runs after build **THEN** it downloads that artifact and rsyncs it to the existing `STAGE_RSYNC_PATH` / `PROD_RSYNC_PATH` (same SSH/rsync secrets as today). - **GIVEN** the **deploy** job **WHEN** it runs after copy **THEN** runtime behaviour matches today: SSH → `docker compose` down/up → `yarn db:up` → `pm2` restart (`card-drop_stage` / `card-drop_prod`), with the same env injection. - **GIVEN** the stage workflow **WHEN** the deploy job is configured **THEN** it uses Forgejo `environment: stage` (not `prod`). - **GIVEN** triggers **WHEN** unchanged branch/tag rules fire **THEN** stage still deploys from `develop`/`hotfix/*`/`release/*`, production from tags. ## Subtasks - [ ] Refactor `.forgejo/workflows/stage.yml` into build / copy / deploy jobs - [ ] Refactor `.forgejo/workflows/production.yml` the same way - [ ] Upload lean Actions artifact in build; download + rsync in copy - [ ] Fix stage deploy `environment: prod` → `stage` - [ ] Verify deploy script still works against lean tree (`db:up`, `pm2 start dist/bot.js`) ## Notes - Runtime stays host **pm2** + compose **MySQL** (no Dockerize bot / Komodo). - Prefer shipping production `node_modules` from the runner (native deps like `canvas`) rather than installing on the host, unless we decide otherwise during implementation. - `rsync --delete` against a lean tree will remove old full-checkout files under the rsync root; ensure no required server-only files live there (or add excludes in a follow-up). - Out of scope: rsync exclude hardening, Komodo, app container image.
Vylpes added this to the 0.11.0 milestone 2026-09-16 18:27:58 +01:00
Smithy-bot commented 2026-10-10 14:33:43 +01:00
Member
Copy link

Planning — criteria locked

Confirmed from owner:

  1. Artifact: Forgejo Actions artifact
  2. Contents: lean runtime package only
  3. Copy target: existing STAGE_RSYNC_PATH / PROD_RSYNC_PATH
  4. Runtime: unchanged (compose MySQL + pm2)
  5. Both stage and production workflows
  6. Drive-by: fix stage deploy environment: prod → stage

Implementation plan

Refactor .forgejo/workflows/stage.yml and production.yml to three jobs:

  1. build — checkout → Node 20 → yarn install --frozen-lockfile → build/test/lint → prepare lean tree (dist/, package.json, yarn.lock, docker-compose.yml, production node_modules/ — prefer prune/reinstall production on the runner so native deps like canvas match the runner) → actions/upload-artifact.
  2. copy (needs: build) — actions/download-artifact → rsync lean tree to existing host path secrets (same rsync action/credentials as today).
  3. deploy (needs: copy) — existing SSH script unchanged in behaviour; stage job uses environment: stage.

Triggers and secret/var injection stay as today. Out of scope: excludes hardening, Komodo, app container.

Moved to needs/estimate.

### Planning — criteria locked Confirmed from owner: 1. **Artifact**: Forgejo Actions artifact 2. **Contents**: lean runtime package only 3. **Copy target**: existing `STAGE_RSYNC_PATH` / `PROD_RSYNC_PATH` 4. **Runtime**: unchanged (compose MySQL + pm2) 5. **Both** stage and production workflows 6. **Drive-by**: fix stage deploy `environment: prod` → `stage` #### Implementation plan Refactor `.forgejo/workflows/stage.yml` and `production.yml` to three jobs: 1. **build** — checkout → Node 20 → `yarn install --frozen-lockfile` → build/test/lint → prepare lean tree (`dist/`, `package.json`, `yarn.lock`, `docker-compose.yml`, production `node_modules/` — prefer prune/reinstall production on the runner so native deps like `canvas` match the runner) → `actions/upload-artifact`. 2. **copy** (`needs: build`) — `actions/download-artifact` → rsync lean tree to existing host path secrets (same rsync action/credentials as today). 3. **deploy** (`needs: copy`) — existing SSH script unchanged in behaviour; stage job uses `environment: stage`. Triggers and secret/var injection stay as today. Out of scope: excludes hardening, Komodo, app container. Moved to **needs/estimate**.
Smithy-bot commented 2026-10-12 12:03:08 +01:00
Member
Copy link

Planning — needs/tests → needs/approval

QA scripts mapped to the acceptance criteria in the issue body. Run after implementation on a branch that triggers stage deploy (e.g. develop); repeat critical checks on a tag production deploy when ready.

Preconditions

  • Forgejo Actions runner can reach stage/prod SSH hosts (same secrets as today).
  • You have read access to workflow run logs and the deployed host (STAGE_RSYNC_PATH / PROD_RSYNC_PATH).
  • Baseline: note current pm2 process name, compose MySQL container, and a smoke Discord command before the change (for deploy regression).

AC1 — Three sequential jobs (build → copy → deploy)

ID Steps Expected
T1.1 Open .forgejo/workflows/stage.yml and production.yml on the implementation branch Each workflow defines build, copy, and deploy jobs
T1.2 Trigger a stage workflow run (push to develop or manual dispatch per repo rules) Run graph shows build completes before copy starts; copy before deploy (needs chain)
T1.3 Repeat inspection for a production workflow run (tag push) Same three-job order

AC2 — Build uploads lean Forgejo Actions artifact

ID Steps Expected
T2.1 On a successful build job log yarn install / build / test / lint steps run as today
T2.2 Inspect build job artifacts in Forgejo UI One (or named) artifact uploaded; not the full git checkout
T2.3 Download artifact locally (or from runner debug) and list contents Includes runtime essentials only: e.g. dist/, package.json, yarn.lock, docker-compose.yml, production node_modules/ — no src/, tests, or dev-only tooling
T2.4 Confirm build job does not rsync to the host Rsync appears only in copy (T3.x)

AC3 — Copy downloads artifact and rsyncs to host path

ID Steps Expected
T3.1 copy job log Downloads the artifact from build
T3.2 copy job log Rsync to STAGE_RSYNC_PATH (stage) or PROD_RSYNC_PATH (prod) using same SSH/rsync secrets as pre-change workflows
T3.3 On host after copy, before deploy completes Rsync root contains lean tree (e.g. dist/, package.json, node_modules/)
T3.4 If --delete is used Obsolete files from old full-checkout deploy are removed; bot still has everything required for yarn db:up and pm2 (no missing dist/bot.js or compose file)

AC4 — Deploy job behaviour unchanged

ID Steps Expected
T4.1 deploy job log (stage) SSH → docker compose down/up → yarn db:up → pm2 restart card-drop_stage with same env injection as before
T4.2 deploy job log (production) Same sequence with card-drop_prod
T4.3 After stage deploy MySQL container healthy; pm2 shows card-drop_stage online
T4.4 Smoke test on Discord (stage) Core bot command works (same as pre-deploy baseline)
T4.5 Migrations yarn db:up succeeds against lean tree on host

AC5 — Stage deploy uses environment: stage

ID Steps Expected
T5.1 Inspect stage.yml deploy job Forgejo environment: stage (not prod)
T5.2 Stage workflow run UI Deploy job associated with stage environment / protection rules

AC6 — Triggers unchanged

ID Steps Expected
T6.1 Push to develop (or configured stage branches: hotfix/*, release/*) Stage workflow triggers
T6.2 Push a version tag per existing production rules Production workflow triggers
T6.3 Unrelated branch push Does not deploy stage/prod unless rules already allowed it

Out of scope (do not fail QA if deferred)

  • Rsync exclude hardening for server-only files under rsync root
  • Komodo / containerized bot image
  • Changing runtime from pm2 + compose MySQL

Sign-off checklist

  • T1–T6 pass on stage; production spot-check on tag when merging.
  • Artifact is lean; copy + deploy succeed; bot smoke test passes.
  • Stage environment fix verified.

Moving to needs/approval for your review before implementation. @Vylpes

## Planning — needs/tests → needs/approval QA scripts mapped to the acceptance criteria in the issue body. Run after implementation on a branch that triggers **stage** deploy (e.g. `develop`); repeat critical checks on a **tag** production deploy when ready. ### Preconditions - Forgejo Actions runner can reach stage/prod SSH hosts (same secrets as today). - You have read access to workflow run logs and the deployed host (`STAGE_RSYNC_PATH` / `PROD_RSYNC_PATH`). - Baseline: note current `pm2` process name, compose MySQL container, and a smoke Discord command before the change (for deploy regression). --- ### AC1 — Three sequential jobs (build → copy → deploy) | ID | Steps | Expected | |----|--------|----------| | T1.1 | Open `.forgejo/workflows/stage.yml` and `production.yml` on the implementation branch | Each workflow defines **build**, **copy**, and **deploy** jobs | | T1.2 | Trigger a stage workflow run (push to `develop` or manual dispatch per repo rules) | Run graph shows **build** completes before **copy** starts; **copy** before **deploy** (`needs` chain) | | T1.3 | Repeat inspection for a production workflow run (tag push) | Same three-job order | --- ### AC2 — Build uploads lean Forgejo Actions artifact | ID | Steps | Expected | |----|--------|----------| | T2.1 | On a successful **build** job log | `yarn install` / build / test / lint steps run as today | | T2.2 | Inspect **build** job artifacts in Forgejo UI | One (or named) artifact uploaded; not the full git checkout | | T2.3 | Download artifact locally (or from runner debug) and list contents | Includes runtime essentials only: e.g. `dist/`, `package.json`, `yarn.lock`, `docker-compose.yml`, production `node_modules/` — **no** `src/`, tests, or dev-only tooling | | T2.4 | Confirm **build** job does **not** rsync to the host | Rsync appears only in **copy** (T3.x) | --- ### AC3 — Copy downloads artifact and rsyncs to host path | ID | Steps | Expected | |----|--------|----------| | T3.1 | **copy** job log | Downloads the artifact from **build** | | T3.2 | **copy** job log | Rsync to `STAGE_RSYNC_PATH` (stage) or `PROD_RSYNC_PATH` (prod) using same SSH/rsync secrets as pre-change workflows | | T3.3 | On host after **copy**, before **deploy** completes | Rsync root contains lean tree (e.g. `dist/`, `package.json`, `node_modules/`) | | T3.4 | If `--delete` is used | Obsolete files from old full-checkout deploy are removed; bot still has everything required for `yarn db:up` and `pm2` (no missing `dist/bot.js` or compose file) | --- ### AC4 — Deploy job behaviour unchanged | ID | Steps | Expected | |----|--------|----------| | T4.1 | **deploy** job log (stage) | SSH → `docker compose` down/up → `yarn db:up` → `pm2` restart `card-drop_stage` with same env injection as before | | T4.2 | **deploy** job log (production) | Same sequence with `card-drop_prod` | | T4.3 | After stage deploy | MySQL container healthy; `pm2` shows `card-drop_stage` online | | T4.4 | Smoke test on Discord (stage) | Core bot command works (same as pre-deploy baseline) | | T4.5 | Migrations | `yarn db:up` succeeds against lean tree on host | --- ### AC5 — Stage deploy uses `environment: stage` | ID | Steps | Expected | |----|--------|----------| | T5.1 | Inspect `stage.yml` **deploy** job | Forgejo `environment: stage` (not `prod`) | | T5.2 | Stage workflow run UI | Deploy job associated with **stage** environment / protection rules | --- ### AC6 — Triggers unchanged | ID | Steps | Expected | |----|--------|----------| | T6.1 | Push to `develop` (or configured stage branches: `hotfix/*`, `release/*`) | Stage workflow triggers | | T6.2 | Push a version tag per existing production rules | Production workflow triggers | | T6.3 | Unrelated branch push | Does **not** deploy stage/prod unless rules already allowed it | --- ### Out of scope (do not fail QA if deferred) - Rsync exclude hardening for server-only files under rsync root - Komodo / containerized bot image - Changing runtime from pm2 + compose MySQL --- ### Sign-off checklist - [ ] T1–T6 pass on stage; production spot-check on tag when merging. - [ ] Artifact is lean; copy + deploy succeed; bot smoke test passes. - [ ] Stage `environment` fix verified. Moving to **needs/approval** for your review before implementation. @Vylpes
Smithy-bot removed their assignment 2026-10-12 12:03:13 +01:00
Vylpes removed their assignment 2026-10-12 12:15:34 +01:00
Vylpes 2026-10-19 21:37:22 +01:00
  • closed this issue
  • added the
    wontfix
         label
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
release/0.11.0
feature/557-upgrade-eslint
release/0.12.0
develop
v0.10.1
v0.10.0
v0.9.4
v0.9.3
v0.9.2
v0.9.1
v0.9.0
v0.8.4
v0.8.3
v0.8.2
v0.8.1
v0.8.0
v0.7.0
v0.6.4
v0.6.3
v0.6.2
v0.6.1
v0.6.0
v0.5.2
v0.5.1
v0.5.0
v0.4.2
v0.4.1
v0.4.0
v0.3.1
v0.3.0
v0.2.1
v0.2.0
v0.1.8
v0.1.7
v0.1.6
v0.1.5
v0.1.4
v0.1.3
v0.1.2
v0.1.1
v0.1.0
Labels
Clear labels
  
blocked

   
duplicate
This issue or pull request already exists

  
high priority
  
needs
approval

  
needs
criteria

  
needs
estimate

  
needs
tests

  
question
More information is needed

  
step
doing

  
step
review

  
step
testing

  
step
todo

  
step
uat

  
type
admin

  
type
alert

  
type
bug

  
type
change

  
type
defect

  
type
dependencies

  
type
development

  
type/documentation
  
type
epic

  
type
idea

  
type
incident

  
type
investigation

  
type
spike

  
type
story

  
type
tests

  
wontfix
This won't be fixed

No labels
blocked
duplicate
high priority
needs
approval
needs
criteria
needs
estimate
needs
tests
question
step
doing
step
review
step
testing
step
todo
step
uat
type
admin
type
alert
type
bug
type
change
type
defect
type
dependencies
type
development
type/documentation
type
epic
type
idea
type
incident
type
investigation
type
spike
type
story
type
tests
wontfix
Milestone
Clear milestone
No items
No milestone
0.11.0
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
External/card-drop#466
No description provided.