feat: backup appwrite
fix: CORS error by adding platforms fix: Update alertmanager listener for event streams
This commit is contained in:
@@ -1,110 +0,0 @@
|
||||
# Session Handoff: Appwrite Bootstrap, Backup, and Bug Fixes
|
||||
**Date:** 2026-03-14
|
||||
**Session Duration:** ~3 hours
|
||||
**Session Focus:** Fix Appwrite console crash, add bootstrap and backup playbooks
|
||||
**Context Usage at Handoff:** ~85%
|
||||
|
||||
---
|
||||
|
||||
## What Was Accomplished
|
||||
|
||||
1. Fixed `_APP_DOMAIN_TARGET_CNAME` null crash → `playbooks/templates/appwrite.env.j2`
|
||||
2. Fixed idempotency: removed `force: true` from compose download → `playbooks/install_appwrite.yml`
|
||||
3. Fixed `appwrite_response_format` undefined error → `playbooks/provision_database.yml`, `playbooks/provision_users.yml`
|
||||
4. Created Appwrite bootstrap playbook → `playbooks/bootstrap_appwrite.yml`
|
||||
5. Created Appwrite backup playbook → `playbooks/backup_appwrite.yml`
|
||||
6. Diagnosed nginx CORS 405 on `apidev.bab.toal.ca` — **not fixed, open**
|
||||
7. Written decision record → `docs/summaries/decisions-2026-03-14-domain-target-fix.md`
|
||||
|
||||
---
|
||||
|
||||
## Exact State of Work in Progress
|
||||
|
||||
- **CORS / nginx**: `apidev.bab.toal.ca` returns HTTP 405 on OPTIONS preflight from nginx/1.20.1. Root cause: nginx config does not pass OPTIONS to backend. `appwrite.toal.ca` works fine. nginx config is managed by `nginxinc.nginx_core` role; no config templates exist in this repo yet.
|
||||
- **backup_appwrite.yml**: Written and structurally correct but **not yet run successfully end-to-end**. Needs a test run and restore verification.
|
||||
|
||||
---
|
||||
|
||||
## Decisions Made This Session
|
||||
|
||||
| Decision | Rationale | Status |
|
||||
|----------|-----------|--------|
|
||||
| `_APP_DOMAIN_TARGET_CNAME` replaces `_APP_DOMAIN_TARGET` | Deprecated since Appwrite 1.7.0; compose `environment:` blocks list the new var, not the old one — old var silently never reached containers | CONFIRMED |
|
||||
| `appwrite_response_format \| default('1.6')` | Var undefined at module_defaults evaluation time; `1.6` is correct format for Appwrite 1.8.x | CONFIRMED |
|
||||
| bootstrap: no account creation task | Appwrite only grants console `owner` role via web UI signup; REST API creates `role: users` which lacks `projects.write` | CONFIRMED |
|
||||
| bootstrap: JWT required for console API | Session cookie alone gives `role: users`; JWT carries team membership claims including `projects.write` | CONFIRMED |
|
||||
| bootstrap: `teamId` fetched from `GET /v1/teams` | Required field in `POST /v1/projects` for Appwrite 1.8.x; discovered from browser network capture | CONFIRMED |
|
||||
| bootstrap: `['$id']` bracket notation | Jinja2 rejects `.$id` — `$` is a special character | CONFIRMED |
|
||||
| bootstrap: `vault_kv2_write` at `kv/oys/bab-appwrite-api-key` | `vault_kv2_put` does not exist; no PATCH operation — dedicated path avoids full-overwrite of other secrets | CONFIRMED |
|
||||
| backup: mysqldump runs while service UP | `--single-transaction` gives consistent InnoDB snapshot; service must be up for `docker compose exec` | CONFIRMED |
|
||||
| backup: `block/rescue/always` | Ensures `systemctl start appwrite` fires even if volume backup fails | CONFIRMED |
|
||||
|
||||
---
|
||||
|
||||
## Key Numbers
|
||||
|
||||
- `appwrite_response_format` default: `1.6`
|
||||
- Vault path for API key: `kv/oys/bab-appwrite-api-key`, key: `appwrite_api_key`
|
||||
- Backup destination: `/var/backups/appwrite/YYYYMMDDTHHMMSS/`
|
||||
- Volumes backed up (8): `appwrite-uploads`, `appwrite-functions`, `appwrite-builds`, `appwrite-sites`, `appwrite-certificates`, `appwrite-config`, `appwrite-cache`, `appwrite-redis`
|
||||
- Volume excluded: `appwrite-mariadb` (covered by mysqldump)
|
||||
|
||||
---
|
||||
|
||||
## Conditional Logic Established
|
||||
|
||||
- IF `appwrite_compose_project` not set THEN `_compose_project` defaults to `basename(appwrite_dir)` = `appwrite` → Docker volume names are `appwrite_appwrite-uploads`, etc.
|
||||
- IF bootstrap re-run THEN second API key created AND Vault entry overwritten — delete old key from console manually
|
||||
- IF backup fails during volume tar THEN `always` block restarts Appwrite — playbook exits failed, partial backup remains in `backup_dir`
|
||||
|
||||
---
|
||||
|
||||
## Files Created or Modified
|
||||
|
||||
| File Path | Action | Description |
|
||||
|-----------|--------|-------------|
|
||||
| `playbooks/templates/appwrite.env.j2` | Modified | Replaced `_APP_DOMAIN_TARGET` with `_APP_DOMAIN_TARGET_CNAME`; added `_APP_DOMAIN_TARGET_CAA` |
|
||||
| `playbooks/install_appwrite.yml` | Modified | Removed `force: true` from `get_url` |
|
||||
| `playbooks/provision_database.yml` | Modified | `appwrite_response_format \| default('1.6')`; fixed long URL line |
|
||||
| `playbooks/provision_users.yml` | Modified | `appwrite_response_format \| default('1.6')` |
|
||||
| `playbooks/bootstrap_appwrite.yml` | Created | Session→JWT→teams→project→API key→Vault |
|
||||
| `playbooks/backup_appwrite.yml` | Created | mysqldump + volume tar + .env, block/rescue/always |
|
||||
| `docs/summaries/decisions-2026-03-14-domain-target-fix.md` | Created | Decision record for domain var fix and idempotency |
|
||||
| `CLAUDE.md` | Modified | Added trailing newline rule |
|
||||
|
||||
---
|
||||
|
||||
## What the NEXT Session Should Do
|
||||
|
||||
1. **Fix nginx CORS** — `apidev.bab.toal.ca` returns 405 on OPTIONS. Load `playbooks/install_nginx.yml`; find where `nginxinc.nginx_core.nginx_config` vars are defined in inventory and add OPTIONS passthrough.
|
||||
2. **Test backup end-to-end** — run `ansible-navigator run playbooks/backup_appwrite.yml --mode stdout`, verify 8 volume tarballs + `mariadb-dump.sql` + `.env` in `/var/backups/appwrite/<timestamp>/`
|
||||
3. **Validate volume name prefix** — run `docker volume ls | grep appwrite` on bab1 to confirm prefix is `appwrite_`
|
||||
|
||||
---
|
||||
|
||||
## Open Questions Requiring User Input
|
||||
|
||||
- [ ] **CORS fix scope**: Should nginx config live in this repo as templates, or managed elsewhere? — impacts `install_nginx.yml` completion
|
||||
- [ ] **Backup retention**: No rotation yet — each run adds a timestamped dir. Add cleanup task?
|
||||
- [ ] **Backup offsite**: 3-2-1 rule — is S3/rsync in scope?
|
||||
|
||||
---
|
||||
|
||||
## Assumptions That Need Validation
|
||||
|
||||
- ASSUMED: Docker Compose project name for volumes is `appwrite` (basename of `/home/ptoal/appwrite`) — validate with `docker volume ls`
|
||||
- ASSUMED: `teams[0]` in bootstrap is always the admin's personal team — valid only if admin has one team
|
||||
|
||||
---
|
||||
|
||||
## What NOT to Re-Read
|
||||
|
||||
- `docs/summaries/handoff-2026-03-14-appwrite-setup-final.md` — superseded; moved to archive
|
||||
|
||||
---
|
||||
|
||||
## Files to Load Next Session
|
||||
|
||||
- `playbooks/install_nginx.yml` — if working on CORS fix
|
||||
- `playbooks/backup_appwrite.yml` — if testing/fixing backup
|
||||
- `docs/context/architecture.md` — for Appwrite API or EDA work
|
||||
@@ -0,0 +1,72 @@
|
||||
# Session Handoff: Appwrite Function DNS Fix
|
||||
**Date:** 2026-03-15
|
||||
**Session Duration:** ~1.5 hours
|
||||
**Session Focus:** Diagnosed and fixed curl error 6 in Appwrite function executor caused by Docker inheriting host search domain
|
||||
**Context Usage at Handoff:** ~60%
|
||||
|
||||
## What Was Accomplished
|
||||
|
||||
1. Diagnosed SMTP auth failure in `appwrite-worker-mails` — deferred (credentials/provider issue, not automation)
|
||||
2. Diagnosed `userinfo` function curl error 6 (CURLE_COULDNT_RESOLVE_HOST) in `openruntimes-executor`
|
||||
3. Identified `_APP_EXECUTOR_RUNTIME_NETWORK` mismatch (`appwrite_runtimes` vs actual Docker network `runtimes`) → fixed in env template default
|
||||
4. Traced root cause to `search mgmt.toal.ca` in container resolv.conf inherited from host → fixed by shortening system hostname from `bab1.mgmt.toal.ca` to `bab1`
|
||||
5. Added pre-flight assertions to `install_appwrite.yml` to prevent recurrence
|
||||
6. Cleaned up ineffective `daemon.json` task added and removed this session
|
||||
|
||||
## Exact State of Work in Progress
|
||||
|
||||
- SMTP authentication failure (`appwrite-worker-mails`): NOT investigated. Separate issue from DNS fix. Deferred.
|
||||
- All DNS/function work: COMPLETE. `userinfo` function confirmed working after hostname change.
|
||||
|
||||
## Decisions Made This Session
|
||||
|
||||
- `_APP_EXECUTOR_RUNTIME_NETWORK` default corrected to `runtimes` BECAUSE the Appwrite docker-compose creates a network named `runtimes` (prefixed by compose project `appwrite`→`appwrite_runtimes`... actually the network is literally named `runtimes` not `appwrite_runtimes`) — STATUS: confirmed, deployed to host
|
||||
- Docker `daemon.json` `"dns-search": []` REJECTED BECAUSE Docker treats empty array as no-op (`# Overrides: []` in container resolv.conf confirms it had no effect)
|
||||
- System hostname shortened to `bab1` BECAUSE FQDN hostname causes NetworkManager to write `search mgmt.toal.ca` into `/etc/resolv.conf`, which Docker inherits into all containers — STATUS: confirmed fix, function working
|
||||
|
||||
## Key Numbers Generated or Discovered This Session
|
||||
|
||||
- Runtime container IP on `runtimes` network: `172.20.0.3`
|
||||
- Executor IP on `runtimes` network: `172.20.0.2`
|
||||
- Executor IP on `appwrite` network: `172.19.0.5`
|
||||
- openruntimes executor image: `openruntimes/executor:0.7.22`
|
||||
- Appwrite version in `install_appwrite.yml`: `1.8.1`
|
||||
- Docker.php error line: 1161 — curl call to `http://{random_32_hex}:3000/`
|
||||
- Runtime hostname format: `bin2hex(random_bytes(16))` = 32-char hex, e.g. `c6991893fe570ce5c669d50ed6e7a985`
|
||||
|
||||
## Conditional Logic Established
|
||||
|
||||
- IF system hostname is FQDN (contains `.`) THEN NetworkManager writes `search <domain>` to `/etc/resolv.conf` AND Docker inherits it into all containers AND Appwrite executor curl calls to runtime containers fail with error 6 BECAUSE musl resolver appends search domain to unqualified names and does not fall back on SERVFAIL
|
||||
- IF `ping {hostname}` resolves but `curl http://{hostname}/` returns error 6 THEN suspect c-ares or `/etc/hosts` vs DNS split — trailing dot in URL (`curl http://{hostname}.:port/`) is a reliable test for whether Docker's embedded DNS has the record
|
||||
- IF `_APP_EXECUTOR_RUNTIME_NETWORK` does not match the actual Docker network name the executor is connected to THEN runtime containers are placed on a different network than the executor and communication fails with error 6
|
||||
|
||||
## Files Created or Modified
|
||||
|
||||
| File Path | Action | Description |
|
||||
|-----------|--------|-------------|
|
||||
| `playbooks/templates/appwrite.env.j2` | Modified | `_APP_EXECUTOR_RUNTIME_NETWORK`, `OPEN_RUNTIMES_NETWORK`, `_APP_FUNCTIONS_RUNTIMES_NETWORK`, `_APP_COMPUTE_RUNTIMES_NETWORK` defaults changed from `appwrite_runtimes` to `runtimes` |
|
||||
| `playbooks/install_appwrite.yml` | Modified | Added pre-flight assertions: hostname must not be FQDN, `/etc/resolv.conf` must have no `search` line. Added explanatory comment block citing the executor curl error 6 failure mode. |
|
||||
|
||||
## What the NEXT Session Should Do
|
||||
|
||||
1. **First**: Read this handoff
|
||||
2. **If SMTP is the goal**: Check `vault_appwrite_smtp_password` value and `appwrite_smtp_username` format against the SMTP provider. The template at `playbooks/templates/appwrite.env.j2` lines 74-78 is correct structurally. The issue is likely credentials or `_APP_SMTP_SECURE` value (`true` string vs `tls`/empty).
|
||||
3. **If function work continues**: The `userinfo` function and DNS are working. Next functional gap is unknown — check Appwrite function logs directly.
|
||||
|
||||
## Open Questions Requiring User Input
|
||||
|
||||
- [ ] SMTP failure (`appwrite-worker-mails` SMTP Error: Could not authenticate) — what provider and were credentials recently rotated? Impacts email delivery for all Appwrite auth flows.
|
||||
|
||||
## Assumptions That Need Validation
|
||||
|
||||
- ASSUMED: Shortening the hostname to `bab1` has no negative side effects on other services on this host (Nginx, AAP connectivity, TLS certs) — validate by checking that `bab1.mgmt.toal.ca` still resolves externally and TLS certs are not hostname-bound to the FQDN system hostname.
|
||||
|
||||
## What NOT to Re-Read
|
||||
|
||||
- `docs/summaries/handoff-2026-03-14-appwrite-bootstrap-backup.md` — archived, superseded by this handoff
|
||||
|
||||
## Files to Load Next Session
|
||||
|
||||
- `playbooks/templates/appwrite.env.j2` — if working on SMTP or any env configuration
|
||||
- `playbooks/install_appwrite.yml` — if adding further host setup tasks
|
||||
- `docs/context/architecture.md` — if working on playbooks or EDA rulebooks
|
||||
Reference in New Issue
Block a user