Files
bab-backend-ansible/docs/archive/handoffs/handoff-2026-03-15-appwrite-function-dns-fix.md

5.3 KiB

Session Handoff: Appwrite Function DNS Fix

Date: 2026-03-15 Session Duration: ~1.5 hours Session Focus: Diagnosed and fixed curl error 6 in Appwrite function executor caused by Docker inheriting host search domain Context Usage at Handoff: ~60%

What Was Accomplished

  1. Diagnosed SMTP auth failure in appwrite-worker-mails — deferred (credentials/provider issue, not automation)
  2. Diagnosed userinfo function curl error 6 (CURLE_COULDNT_RESOLVE_HOST) in openruntimes-executor
  3. Identified _APP_EXECUTOR_RUNTIME_NETWORK mismatch (appwrite_runtimes vs actual Docker network runtimes) → fixed in env template default
  4. Traced root cause to search mgmt.toal.ca in container resolv.conf inherited from host → fixed by shortening system hostname from bab1.mgmt.toal.ca to bab1
  5. Added pre-flight assertions to install_appwrite.yml to prevent recurrence
  6. Cleaned up ineffective daemon.json task added and removed this session

Exact State of Work in Progress

  • SMTP authentication failure (appwrite-worker-mails): NOT investigated. Separate issue from DNS fix. Deferred.
  • All DNS/function work: COMPLETE. userinfo function confirmed working after hostname change.

Decisions Made This Session

  • _APP_EXECUTOR_RUNTIME_NETWORK default corrected to runtimes BECAUSE the Appwrite docker-compose creates a network named runtimes (prefixed by compose project appwriteappwrite_runtimes... actually the network is literally named runtimes not appwrite_runtimes) — STATUS: confirmed, deployed to host
  • Docker daemon.json "dns-search": [] REJECTED BECAUSE Docker treats empty array as no-op (# Overrides: [] in container resolv.conf confirms it had no effect)
  • System hostname shortened to bab1 BECAUSE FQDN hostname causes NetworkManager to write search mgmt.toal.ca into /etc/resolv.conf, which Docker inherits into all containers — STATUS: confirmed fix, function working

Key Numbers Generated or Discovered This Session

  • Runtime container IP on runtimes network: 172.20.0.3
  • Executor IP on runtimes network: 172.20.0.2
  • Executor IP on appwrite network: 172.19.0.5
  • openruntimes executor image: openruntimes/executor:0.7.22
  • Appwrite version in install_appwrite.yml: 1.8.1
  • Docker.php error line: 1161 — curl call to http://{random_32_hex}:3000/
  • Runtime hostname format: bin2hex(random_bytes(16)) = 32-char hex, e.g. c6991893fe570ce5c669d50ed6e7a985

Conditional Logic Established

  • IF system hostname is FQDN (contains .) THEN NetworkManager writes search <domain> to /etc/resolv.conf AND Docker inherits it into all containers AND Appwrite executor curl calls to runtime containers fail with error 6 BECAUSE musl resolver appends search domain to unqualified names and does not fall back on SERVFAIL
  • IF ping {hostname} resolves but curl http://{hostname}/ returns error 6 THEN suspect c-ares or /etc/hosts vs DNS split — trailing dot in URL (curl http://{hostname}.:port/) is a reliable test for whether Docker's embedded DNS has the record
  • IF _APP_EXECUTOR_RUNTIME_NETWORK does not match the actual Docker network name the executor is connected to THEN runtime containers are placed on a different network than the executor and communication fails with error 6

Files Created or Modified

File Path Action Description
playbooks/templates/appwrite.env.j2 Modified _APP_EXECUTOR_RUNTIME_NETWORK, OPEN_RUNTIMES_NETWORK, _APP_FUNCTIONS_RUNTIMES_NETWORK, _APP_COMPUTE_RUNTIMES_NETWORK defaults changed from appwrite_runtimes to runtimes
playbooks/install_appwrite.yml Modified Added pre-flight assertions: hostname must not be FQDN, /etc/resolv.conf must have no search line. Added explanatory comment block citing the executor curl error 6 failure mode.

What the NEXT Session Should Do

  1. First: Read this handoff
  2. If SMTP is the goal: Check vault_appwrite_smtp_password value and appwrite_smtp_username format against the SMTP provider. The template at playbooks/templates/appwrite.env.j2 lines 74-78 is correct structurally. The issue is likely credentials or _APP_SMTP_SECURE value (true string vs tls/empty).
  3. If function work continues: The userinfo function and DNS are working. Next functional gap is unknown — check Appwrite function logs directly.

Open Questions Requiring User Input

  • SMTP failure (appwrite-worker-mails SMTP Error: Could not authenticate) — what provider and were credentials recently rotated? Impacts email delivery for all Appwrite auth flows.

Assumptions That Need Validation

  • ASSUMED: Shortening the hostname to bab1 has no negative side effects on other services on this host (Nginx, AAP connectivity, TLS certs) — validate by checking that bab1.mgmt.toal.ca still resolves externally and TLS certs are not hostname-bound to the FQDN system hostname.

What NOT to Re-Read

  • docs/summaries/handoff-2026-03-14-appwrite-bootstrap-backup.md — archived, superseded by this handoff

Files to Load Next Session

  • playbooks/templates/appwrite.env.j2 — if working on SMTP or any env configuration
  • playbooks/install_appwrite.yml — if adding further host setup tasks
  • docs/context/architecture.md — if working on playbooks or EDA rulebooks