Update storage and Keycloak config

.ansible-lint

@@ -23,10 +23,6 @@ warn_list:
   - fqcn[action-core]
   - no-changed-when
 
-# Rules to skip entirely during initial adoption
-skip_list:
-  - role-name  # toal-common doesn't follow FQCN yet
-
 # Use progressive mode: only flag new violations on changed files
 # (useful for gradual adoption in existing projects)
 # progressive: true

CLAUDE.md

@@ -0,0 +1,42 @@
+# CLAUDE.md
+
+## Session Start
+
+Read the latest handoff in docs/summaries/ if one exists. Load only the files that handoff references — not all summaries. If no handoff exists, ask: what is the project, what type of work, what is the target deliverable.
+
+Before starting work, state: what you understand the project state to be, what you plan to do this session, and any open questions.
+
+## Identity
+
+You work with Pat, a Senior Solutions Architect at Red Hat building automation for a HomeLab.
+
+## Rules
+
+1. Do not mix unrelated project contexts in one session.
+2. Write state to disk, not conversation. After completing meaningful work, write a summary to docs/summaries/ using templates from templates/claude-templates.md. Include: decisions with rationale, exact numbers, file paths, open items.
+3. Before compaction or session end, write to disk: every number, every decision with rationale, every open question, every file path, exact next action.
+4. When switching work types (research → writing → review), write a handoff to docs/summaries/handoff-[date]-[topic].md and suggest a new session.
+5. Do not silently resolve open questions. Mark them OPEN or ASSUMED.
+6. Do not bulk-read documents. Process one at a time: read, summarize to disk, release from context before reading next. For the detailed protocol, read docs/context/processing-protocol.md.
+7. Sub-agent returns must be structured, not free-form prose. Use output contracts from templates/claude-templates.md.
+
+## Where Things Live
+
+- templates/claude-templates.md — summary, handoff, decision, analysis, task, output contract templates (read on demand)
+- docs/summaries/ — active session state (latest handoff + project brief + decision records + source summaries)
+- docs/context/ — reusable domain knowledge, loaded only when relevant to the current task
+  - processing-protocol.md — full document processing steps
+  - archive-rules.md — summary lifecycle and file archival rules
+- playbooks/ -- Main ansible playbooks
+- roles/ -- Both custom, and external Ansible roles
+- collections/ -- should only contain requirements.yml
+- docs/archive/ — processed raw files. Do not read unless explicitly told.
+- output/deliverables/ — final outputs
+
+## Error Recovery
+
+If context degrades or auto-compact fires unexpectedly: write current state to docs/summaries/recovery-[date].md, tell the user what may have been lost, suggest a fresh session.
+
+## Before Delivering Output
+
+Verify: exact numbers preserved, open questions marked OPEN, output matches what was requested (not assumed), claims backed by specific data, output consistent with stored decisions in docs/context/, summary written to disk for this session's work.

ansible.cfg

@@ -1,6 +1,6 @@
 [defaults]
 # Inventory - override with -i or ANSIBLE_INVENTORY env var
-inventory              = /home/ptoal/Dev/inventories/toallab-inventory
+inventory              = /home/ptoal/Dev/inventories/toallab-inventory/static.yml
 
 # Role and collection paths
 roles_path             = roles

collections/requirements.yml

@@ -12,3 +12,5 @@ collections:
     source: https://github.com/O-X-L/ansible-opnsense
     type: git
     version: latest
+  - name: middleware_automation.keycloak
+  - name: infra.aap_configuration

docs/summaries/2026-02-26-aap-keycloak-oidc.md

@@ -0,0 +1,54 @@
+# Session Summary: AAP Keycloak OIDC Configuration
+Date: 2026-02-26
+
+## Work Done
+Added Keycloak OIDC authentication support for AAP 2.6 using the correct approach:
+`infra.aap_configuration.gateway_authenticators` (AAP Gateway API) instead of CR extra_settings (wrong for 2.6).
+
+## Files Changed
+- `collections/requirements.yml` — Added `infra.aap_configuration`
+- `playbooks/deploy_aap.yml` — Full rewrite:
+  - Play 0 (`aap_configure_keycloak`): Creates Keycloak OIDC client with correct callback URI `/accounts/profile/callback/`
+  - Play 1: Unchanged (installs AAP via `aap_operator` role)
+  - Play 2 (`aap_configure_oidc`): Fetches admin password from K8s secret, calls `infra.aap_configuration.gateway_authenticators`
+- `roles/aap_operator/defaults/main.yml` — Removed OIDC vars (not role responsibility)
+- `roles/aap_operator/meta/argument_specs.yml` — Removed OIDC var docs
+- `roles/aap_operator/tasks/main.yml` — Removed OIDC include task (was wrong approach)
+- `roles/aap_operator/tasks/configure_oidc.yml` — Replaced with redirect comment
+
+## Key Decisions
+- **OIDC must be configured via AAP Gateway API** (not CR extra_settings). AAP 2.5+ Gateway uses Django-based auth with `ansible_base.authentication` plugins.
+- **authenticator type**: `ansible_base.authentication.authenticator_plugins.generic_oidc`
+- **Callback URL**: `{aap_gateway_url}/accounts/profile/callback/` (not `/social/complete/oidc/`)
+- **Admin password**: Fetched dynamically from K8s secret `{platform_name}-admin-password` (not stored separately in vault)
+- **OIDC not in `aap_operator` role**: Kept as a separate playbook play (post-install concern)
+
+## Variables Required in `aap` host_vars
+```yaml
+aap_gateway_url: "https://aap.apps.<cluster>.<domain>"
+aap_oidc_issuer: "https://keycloak.toal.ca/realms/<realm>"
+aap_oidc_client_id: aap   # optional, default: aap
+```
+
+## Vault Variables
+```
+vault_aap_oidc_client_secret  — OIDC client secret from Keycloak
+vault_aap_deployer_token      — K8s SA token (already required)
+vault_keycloak_admin_password — required for Play 0
+```
+
+## Usage
+```bash
+# Step 1: Create Keycloak client (once, idempotent)
+ansible-navigator run playbooks/deploy_aap.yml --tags aap_configure_keycloak
+
+# Step 2: Deploy AAP
+ansible-navigator run playbooks/deploy_aap.yml
+
+# Step 3: Register OIDC authenticator in AAP Gateway
+ansible-navigator run playbooks/deploy_aap.yml --tags aap_configure_oidc
+```
+
+## Open Items
+- ASSUMED: `infra.aap_configuration` + its dependency `ansible.platform` are available or installable in `aap.toal.ca/ee-demo:latest`. If not, a custom EE rebuild is needed.
+- The `aap-deployer` SA has `get` on secrets in `aap` namespace — confirmed via RBAC in `deploy_openshift.yml` Play 9.

playbooks/bootstrap.yml

@@ -25,12 +25,6 @@
       state: present
       when: ansible_os_family == "RedHat"
 
-- name: Set up Basic Lab Packages
-  hosts: all
-  become: yes
-  roles:
-    - role: toal-common
-
 - name: Packages
   hosts: all
   become: yes

playbooks/cvpublish.yml

@@ -1,46 +0,0 @@
-- name: Publish CVs
-  hosts: satellite1.mgmt.toal.ca
-  vars:
-    sat_env_name: Library
-    sat_org: Toal.ca
-    sat_publish_description: Automated CV Update
-
-  tasks:
-  - name: Pre-tasks | Find all CVs
-    redhat.satellite.resource_info:
-      username: "{{ satellite_admin_user }}"
-      password: "{{ satellite_admin_pass }}"
-      server_url: "{{ satellite_url }}"
-      organization: "{{ sat_org }}"
-      resource: content_views
-      validate_certs: no
-    register: raw_list_cvs
-
-  - name: Pre-tasks | Get resource information
-    set_fact:
-      list_all_cvs: "{{ raw_list_cvs['resources'] | json_query(jmesquery) | list }}"
-    vars: 
-      jmesquery: "[*].{name: name, composite: composite, id: id}"
-    
-  - name: Pre-tasks | Extract list of content views
-    set_fact:
-      sat6_content_views_list: "{{ sat6_content_views_list|default([]) }} + ['{{ item.name }}' ]"
-    loop: "{{ list_all_cvs | reject('search', 'Default Organization View') | list }}"
-    when: item.composite == false 
-
-  - name: Publish content
-    redhat.satellite.content_view_version:
-      username: "{{ satellite_admin_user }}"
-      password: "{{ satellite_admin_pass }}"
-      server_url: "{{ satellite_url }}"
-      organization: "{{ sat_org }}"
-      content_view: "{{ item }}"
-      validate_certs: no
-      description: "{{ sat_publish_description }}"
-      lifecycle_environments:
-        - Library
-        - "{{ sat_env_name }}"
-    loop: "{{ sat6_content_views_list | list }}"
-    loop_control:
-      loop_var: "item"
-    register: cv_publish_sleeper

playbooks/deploy_aap.yml

@@ -0,0 +1,219 @@
+---
+# Deploy Ansible Automation Platform on OpenShift
+#
+# Authenticates via the aap-deployer ServiceAccount token (not kubeadmin).
+# The token is stored in 1Password and loaded via vault_aap_deployer_token.
+#
+# Prerequisites:
+#   - OpenShift cluster deployed (deploy_openshift.yml)
+#   - aap-deployer ServiceAccount provisioned:
+#       ansible-navigator run playbooks/deploy_openshift.yml --tags sno_deploy_service_accounts
+#   - SA token saved to 1Password as vault_aap_deployer_token
+#
+# Keycloak OIDC prerequisites (--tags aap_configure_keycloak,aap_configure_oidc):
+#   - Keycloak realm exists (configured via deploy_openshift.yml)
+#   - vault_aap_oidc_client_secret in 1Password (or it will be generated and displayed)
+#   - In host_vars for the aap host:
+#       aap_gateway_url: "https://aap.apps.<cluster>.<domain>"
+#       aap_oidc_client_id: aap
+#       aap_oidc_issuer: "https://keycloak.example.com/realms/<realm>"
+#       aap_oidc_public_key: "<RS256 public key from Keycloak realm Keys tab>"
+#
+# Play order:
+#   Play 0: aap_configure_keycloak  — Create Keycloak OIDC client for AAP Gateway
+#   Play 1: (default)               — Install AAP via aap_operator role
+#   Play 2: aap_configure_oidc      — Configure OIDC Authentication Method in AAP Gateway
+#
+# Usage:
+#   ansible-navigator run playbooks/deploy_aap.yml
+#   ansible-navigator run playbooks/deploy_aap.yml --tags aap_configure_keycloak
+#   ansible-navigator run playbooks/deploy_aap.yml --tags aap_configure_oidc
+#   ansible-navigator run playbooks/deploy_aap.yml --tags aap_configure_keycloak,aap_configure_oidc
+
+# ---------------------------------------------------------------------------
+# Play 0: Create Keycloak OIDC client for AAP (optional)
+# Runs on openshift hosts to access keycloak_url/keycloak_realm host vars.
+# Creates the OIDC client in Keycloak with the correct AAP Gateway callback URI.
+# ---------------------------------------------------------------------------
+- name: Configure Keycloak OIDC client for AAP
+  hosts: openshift
+  gather_facts: false
+  connection: local
+
+  tags:
+    - never
+    - aap_configure_keycloak
+
+  vars:
+    __aap_keycloak_api_url: "{{ keycloak_url }}{{ keycloak_context | default('') }}"
+    __aap_oidc_client_id: "{{ aap_oidc_client_id | default('aap') }}"
+    # AAP operator generates the Gateway route as {platform_name}-{namespace}.apps.{cluster}.{domain}
+    # e.g. platform 'aap' in namespace 'aap' → aap-aap.apps.openshift.toal.ca
+    __aap_platform_name: "{{ aap_operator_platform_name | default('aap') }}"
+    __aap_namespace: "{{ aap_operator_namespace | default('aap') }}"
+    __aap_oidc_redirect_uris:
+      - "https://{{ __aap_platform_name }}-{{ __aap_namespace }}.apps.{{ ocp_cluster_name }}.{{ ocp_base_domain }}/accounts/profile/callback/"
+
+  module_defaults:
+    middleware_automation.keycloak.keycloak_client:
+      auth_client_id: admin-cli
+      auth_keycloak_url: "{{ __aap_keycloak_api_url }}"
+      auth_realm: master
+      auth_username: "{{ keycloak_admin_user }}"
+      auth_password: "{{ vault_keycloak_admin_password }}"
+      validate_certs: "{{ keycloak_validate_certs | default(true) }}"
+
+  tasks:
+    - name: Set AAP OIDC client secret (vault value or generated)
+      ansible.builtin.set_fact:
+        __aap_oidc_client_secret: "{{ vault_aap_oidc_client_secret | default(lookup('community.general.random_string', length=32, special=false)) }}"
+        __aap_oidc_secret_generated: "{{ vault_aap_oidc_client_secret is not defined }}"
+      no_log: true
+
+    - name: Create AAP OIDC client in Keycloak
+      middleware_automation.keycloak.keycloak_client:
+        realm: "{{ keycloak_realm }}"
+        client_id: "{{ __aap_oidc_client_id }}"
+        name: "Ansible Automation Platform"
+        description: "OIDC client for AAP Gateway on {{ ocp_cluster_name }}.{{ ocp_base_domain }}"
+        enabled: true
+        protocol: openid-connect
+        public_client: false
+        standard_flow_enabled: true
+        implicit_flow_enabled: false
+        direct_access_grants_enabled: false
+        service_accounts_enabled: false
+        secret: "{{ __aap_oidc_client_secret }}"
+        redirect_uris: "{{ __aap_oidc_redirect_uris }}"
+        web_origins:
+          - "+"
+        protocol_mappers:
+          - name: groups
+            protocol: openid-connect
+            protocolMapper: oidc-group-membership-mapper
+            config:
+              full.path: "false"
+              id.token.claim: "true"
+              access.token.claim: "true"
+              userinfo.token.claim: "true"
+              claim.name: groups
+        state: present
+      no_log: "{{ keycloak_no_log | default(true) }}"
+
+    - name: Display generated client secret (save this to vault!)
+      ansible.builtin.debug:
+        msg:
+          - "*** GENERATED AAP OIDC CLIENT SECRET — SAVE THIS TO VAULT ***"
+          - "vault_aap_oidc_client_secret: {{ __aap_oidc_client_secret }}"
+          - ""
+          - "Save to 1Password and reference as vault_aap_oidc_client_secret."
+      when: __aap_oidc_secret_generated | bool
+
+    - name: Display Keycloak AAP OIDC configuration summary
+      ansible.builtin.debug:
+        msg:
+          - "Keycloak AAP OIDC client configured:"
+          - "  Realm    : {{ keycloak_realm }}"
+          - "  Client   : {{ __aap_oidc_client_id }}"
+          - "  Issuer   : {{ __aap_keycloak_api_url }}/realms/{{ keycloak_realm }}"
+          - "  Redirect : {{ __aap_oidc_redirect_uris | join(', ') }}"
+          - ""
+          - "Set in host_vars for the aap host:"
+          - "  aap_gateway_url: https://{{ __aap_platform_name }}-{{ __aap_namespace }}.apps.{{ ocp_cluster_name }}.{{ ocp_base_domain }}"
+          - "  aap_oidc_issuer: {{ __aap_keycloak_api_url }}/realms/{{ keycloak_realm }}"
+          - ""
+          - "Then run: --tags aap_configure_oidc to register the authenticator in AAP."
+        verbosity: 1
+
+# ---------------------------------------------------------------------------
+# Play 1: Install Ansible Automation Platform
+# ---------------------------------------------------------------------------
+- name: Install Ansible Automation Platform
+  hosts: aap
+  gather_facts: false
+  connection: local
+
+  pre_tasks:
+    - name: Verify aap-deployer token is available
+      ansible.builtin.assert:
+        that:
+          - vault_aap_deployer_token is defined
+          - vault_aap_deployer_token | length > 0
+        fail_msg: >-
+          vault_aap_deployer_token is not set. Provision the ServiceAccount with:
+          ansible-navigator run playbooks/deploy_openshift.yml --tags sno_deploy_service_accounts
+          Then save the displayed token to 1Password as vault_aap_deployer_token.
+
+#  environment:
+#    K8S_AUTH_HOST: "{{ aap_k8s_api_url }}"
+#    K8S_AUTH_API_KEY: "{{ vault_aap_deployer_token }}"
+
+  roles:
+    - role: aap_operator
+
+# ---------------------------------------------------------------------------
+# Play 2: Configure Keycloak OIDC Authentication Method in AAP Gateway (optional)
+# Uses infra.aap_configuration.gateway_authenticators to register the OIDC
+# provider via the AAP Gateway API. Run after Play 1 (AAP must be Running).
+#
+# Requires in host_vars for the aap host:
+#   aap_gateway_url:      "https://aap.apps.<cluster>.<domain>"
+#   aap_oidc_issuer:      "https://keycloak.example.com/realms/<realm>"
+#   aap_oidc_client_id:   aap   (optional, default: aap)
+#   aap_oidc_public_key:  "<RS256 public key from Keycloak realm Keys tab>"
+# Vault:
+#   vault_aap_oidc_client_secret  — OIDC client secret from Keycloak
+# ---------------------------------------------------------------------------
+- name: Configure Keycloak OIDC Authentication in AAP Gateway
+  hosts: aap
+  gather_facts: false
+  connection: local
+
+  tags:
+    - never
+    - aap_configure_oidc
+
+  vars:
+    __aap_namespace: "{{ aap_operator_namespace | default('aap') }}"
+    __aap_platform_name: "{{ aap_operator_platform_name | default('aap') }}"
+
+  environment:
+    K8S_AUTH_HOST: "{{ aap_k8s_api_url }}"
+    K8S_AUTH_API_KEY: "{{ vault_aap_deployer_token }}"
+
+  pre_tasks:
+    - name: Fetch AAP admin password from K8s secret
+      kubernetes.core.k8s_info:
+        api_version: v1
+        kind: Secret
+        namespace: "{{ __aap_namespace }}"
+        name: "{{ __aap_platform_name }}-admin-password"
+      register: __aap_admin_secret
+      no_log: false
+
+    - name: Set AAP admin password fact
+      ansible.builtin.set_fact:
+        __aap_admin_password: "{{ __aap_admin_secret.resources[0].data.password | b64decode }}"
+      no_log: true
+
+  tasks:
+    - name: Configure Keycloak OIDC authenticator in AAP Gateway
+      ansible.builtin.include_role:
+        name: infra.aap_configuration.gateway_authenticators
+      vars:
+        aap_hostname: "{{ aap_gateway_url }}"
+        aap_username: "{{ aap_operator_admin_user | default('admin') }}"
+        aap_password: "{{ __aap_admin_password }}"
+        gateway_authenticators:
+          - name: Keycloak
+            type: ansible_base.authentication.authenticator_plugins.keycloak
+            slug: keycloak
+            enabled: true
+            configuration:
+              KEY: "{{ aap_oidc_client_id | default('aap') }}"
+              SECRET: "{{ vault_aap_oidc_client_secret }}"
+              PUBLIC_KEY: "{{ aap_oidc_public_key }}"
+              ACCESS_TOKEN_URL: "{{ aap_oidc_issuer }}/protocol/openid-connect/token"
+              AUTHORIZATION_URL: "{{ aap_oidc_issuer }}/protocol/openid-connect/auth"
+              GROUPS_CLAIM: "groups"
+            state: present

playbooks/deploy_openshift.yml

@@ -1,37 +1,51 @@
 ---
-# Deploy Single Node OpenShift (SNO) on Proxmox
+# Deploy and configure Single Node OpenShift (SNO) on Proxmox
 #
 # Prerequisites:
 #   ansible-galaxy collection install -r collections/requirements.yml
-#   openshift-install is downloaded automatically during the sno play
+#   openshift-install is downloaded automatically during the sno_deploy_install play
 #
 # Inventory requirements:
 #   sno.openshift.toal.ca  - in 'openshift' group
 #     host_vars: ocp_cluster_name, ocp_base_domain, ocp_version, sno_ip,
-#                sno_gateway, sno_nameserver, sno_prefix_length, sno_vm_name,
-#                sno_bridge, sno_vlan, proxmox_node, ...
-#     secrets:  vault_ocp_pull_secret (Red Hat pull secret JSON string)
-#   proxmox_api            - inventory host (ansible_host: proxmox.lab.toal.ca, ansible_port: 443)
-#                            Used as api_host / api_port source for community.proxmox modules
-#   proxmox_host           - inventory host (ansible_host: pve1.lab.toal.ca, ansible_connection: ssh)
-#                            delegate_to target for qm and file operations
+#                sno_gateway, sno_nameserver, sno_prefix_length, sno_machine_network,
+#                sno_vm_name, sno_vnet, sno_storage_ip, sno_storage_ip_prefix_length,
+#                sno_storage_vnet, proxmox_node, keycloak_url, keycloak_realm,
+#                oidc_admin_groups, sno_deploy_letsencrypt_email, ...
+#     secrets:  vault_ocp_pull_secret, vault_keycloak_admin_password,
+#               vault_oidc_client_secret (optional)
+#     optional: ocp_kubeconfig (defaults to ~/.kube/config; set to
+#               sno_install_dir/auth/kubeconfig for fresh installs)
+#   proxmox_api            - inventory host (ansible_host, ansible_port)
+#   proxmox_host           - inventory host (ansible_host, ansible_connection: ssh)
 #   gate.toal.ca           - in 'opnsense' group
 #     host_vars: opnsense_host, opnsense_api_key, opnsense_api_secret,
 #                opnsense_api_port, haproxy_public_ip
 #   group_vars/all:        dme_account_key, dme_account_secret
 #
 # Play order (intentional — DNS must precede VM boot):
-#   Play 1: proxmox  — Create SNO VM
-#   Play 2: opnsense — Configure OPNsense local DNS overrides (api/api-int/apps)
-#   Play 3: dns      — Configure public DNS records in DNS Made Easy
-#   Play 4: sno      — Generate ISO, boot VM, wait for install
+#   Play 1: sno_deploy_vm       — Create SNO VM
+#   Play 2: opnsense            — Configure OPNsense local DNS overrides
+#   Play 3: dns                 — Configure public DNS records in DNS Made Easy
+#   Play 4: sno_deploy_install  — Generate ISO, boot VM, wait for install
+#   Play 5: keycloak            — Configure Keycloak OIDC client
+#   Play 6: sno_deploy_oidc / sno_deploy_certmanager / sno_deploy_delete_kubeadmin
+#   Play 7: sno_deploy_lvms            — Install LVM Storage for persistent volumes
+#   Play 8: sno_deploy_nfs             — Deploy in-cluster NFS provisioner (RWX StorageClass)
+#   Play 9: sno_deploy_service_accounts — Provision ServiceAccounts for app deployers
+#
+# AAP deployment is in a separate playbook: deploy_aap.yml
 #
 # Usage:
-#   ansible-playbook playbooks/deploy_openshift.yml
-#   ansible-playbook playbooks/deploy_openshift.yml --tags proxmox
-#   ansible-playbook playbooks/deploy_openshift.yml --tags sno
-#   ansible-playbook playbooks/deploy_openshift.yml --tags dns,opnsense
-#   ansible-playbook playbooks/deploy_openshift.yml --tags opnsense,sno
+#   ansible-navigator run playbooks/deploy_openshift.yml
+#   ansible-navigator run playbooks/deploy_openshift.yml --tags sno_deploy_vm
+#   ansible-navigator run playbooks/deploy_openshift.yml --tags sno_deploy_install
+#   ansible-navigator run playbooks/deploy_openshift.yml --tags opnsense,dns
+#   ansible-navigator run playbooks/deploy_openshift.yml --tags keycloak,sno_deploy_oidc
+#   ansible-navigator run playbooks/deploy_openshift.yml --tags sno_deploy_certmanager
+#   ansible-navigator run playbooks/deploy_openshift.yml --tags sno_deploy_lvms
+#   ansible-navigator run playbooks/deploy_openshift.yml --tags sno_deploy_nfs
+#   ansible-navigator run playbooks/deploy_openshift.yml --tags sno_deploy_service_accounts
 
 # ---------------------------------------------------------------------------
 # Play 1: Create SNO VM in Proxmox
@@ -40,10 +54,13 @@
   hosts: sno.openshift.toal.ca
   gather_facts: false
   connection: local
+  tags: sno_deploy_vm
 
-  roles:
-    - role: proxmox_sno_vm
-      tags: proxmox
+  tasks:
+    - name: Create VM
+      ansible.builtin.include_role:
+        name: sno_deploy
+        tasks_from: create_vm.yml
 
 # ---------------------------------------------------------------------------
 # Play 2: Configure OPNsense - Local DNS Overrides
@@ -63,50 +80,27 @@
       ssl_verify: "{{ opnsense_ssl_verify | default(false) }}"
       api_port: "{{ opnsense_api_port | default(omit) }}"
 
+  vars:
+    __deploy_ocp_cluster_name: "{{ hostvars['sno.openshift.toal.ca']['ocp_cluster_name'] }}"
+    __deploy_ocp_base_domain: "{{ hostvars['sno.openshift.toal.ca']['ocp_base_domain'] }}"
+    __deploy_sno_ip: "{{ hostvars['sno.openshift.toal.ca']['sno_ip'] }}"
+
   tags: opnsense
 
-  tasks:
-
-    - name: Add Unbound host override for OCP API
-      oxlorg.opnsense.unbound_host:
-        hostname: "api.{{ ocp_cluster_name }}"
-        domain: "{{ ocp_base_domain }}"
-        value: "{{ sno_ip }}"
-        match_fields:
-          - hostname
-          - domain
-        state: present
-      delegate_to: localhost
-      vars:
-        ocp_cluster_name: "{{ hostvars['sno.openshift.toal.ca']['ocp_cluster_name'] }}"
-        ocp_base_domain: "{{ hostvars['sno.openshift.toal.ca']['ocp_base_domain'] }}"
-        sno_ip: "{{ hostvars['sno.openshift.toal.ca']['sno_ip'] }}"
-
-    - name: Add Unbound host override for OCP API internal
-      oxlorg.opnsense.unbound_host:
-        hostname: "api-int.{{ ocp_cluster_name }}"
-        domain: "{{ ocp_base_domain }}"
-        value: "{{ sno_ip }}"
-        match_fields:
-          - hostname
-          - domain
-        state: present
-      delegate_to: localhost
-      vars:
-        ocp_cluster_name: "{{ hostvars['sno.openshift.toal.ca']['ocp_cluster_name'] }}"
-        ocp_base_domain: "{{ hostvars['sno.openshift.toal.ca']['ocp_base_domain'] }}"
-        sno_ip: "{{ hostvars['sno.openshift.toal.ca']['sno_ip'] }}"
-
-    - name: Forward apps wildcard domain to SNO ingress
-      oxlorg.opnsense.unbound_forward:
-        domain: "apps.{{ ocp_cluster_name }}.{{ ocp_base_domain }}"
-        target: "{{ sno_ip }}"
-        state: present
-      delegate_to: localhost
-      vars:
-        ocp_cluster_name: "{{ hostvars['sno.openshift.toal.ca']['ocp_cluster_name'] }}"
-        ocp_base_domain: "{{ hostvars['sno.openshift.toal.ca']['ocp_base_domain'] }}"
-        sno_ip: "{{ hostvars['sno.openshift.toal.ca']['sno_ip'] }}"
+  roles:
+    - role: opnsense_dns_override
+      opnsense_dns_override_entries:
+        - hostname: "api.{{ __deploy_ocp_cluster_name }}"
+          domain: "{{ __deploy_ocp_base_domain }}"
+          value: "{{ __deploy_sno_ip }}"
+          type: host
+        - hostname: "api-int.{{ __deploy_ocp_cluster_name }}"
+          domain: "{{ __deploy_ocp_base_domain }}"
+          value: "{{ __deploy_sno_ip }}"
+          type: host
+        - domain: "apps.{{ __deploy_ocp_cluster_name }}.{{ __deploy_ocp_base_domain }}"
+          value: "{{ __deploy_sno_ip }}"
+          type: forward
 
 # ---------------------------------------------------------------------------
 # Play 3: Configure Public DNS Records in DNS Made Easy
@@ -116,290 +110,255 @@
   gather_facts: false
   connection: local
 
+  vars:
+    __deploy_public_ip: "{{ hostvars['gate.toal.ca']['haproxy_public_ip'] }}"
+
   tags: dns
 
-  tasks:
-
-    - name: Create A record for OpenShift API endpoint
-      community.general.dnsmadeeasy:
-        account_key: "{{ dme_account_key }}"
-        account_secret: "{{ dme_account_secret }}"
-        domain: "{{ ocp_base_domain }}"
-        record_name: "api.{{ ocp_cluster_name }}"
-        record_type: A
-        record_value: "{{ hostvars['gate.toal.ca']['haproxy_public_ip'] }}"
-        record_ttl: "{{ ocp_dns_ttl }}"
-        port: 443
-        protocol: HTTPS
-        state: present
-
-    - name: Create A record for OpenShift apps wildcard
-      community.general.dnsmadeeasy:
-        account_key: "{{ dme_account_key }}"
-        account_secret: "{{ dme_account_secret }}"
-        domain: "{{ ocp_base_domain }}"
-        record_name: "*.apps.{{ ocp_cluster_name }}"
-        record_type: A
-        record_value: "{{ hostvars['gate.toal.ca']['haproxy_public_ip'] }}"
-        record_ttl: "{{ ocp_dns_ttl }}"
-        port: 443
-        protocol: HTTPS
-        state: present
+  roles:
+    - role: dnsmadeeasy_record
+      dnsmadeeasy_record_account_key: "{{ dme_account_key }}"
+      dnsmadeeasy_record_account_secret: "{{ dme_account_secret }}"
+      dnsmadeeasy_record_entries:
+        - domain: "{{ ocp_base_domain }}"
+          record_name: "api.{{ ocp_cluster_name }}"
+          record_type: A
+          record_value: "{{ __deploy_public_ip }}"
+          record_ttl: "{{ ocp_dns_ttl }}"
+        - domain: "{{ ocp_base_domain }}"
+          record_name: "*.apps.{{ ocp_cluster_name }}"
+          record_type: A
+          record_value: "{{ __deploy_public_ip }}"
+          record_ttl: "{{ ocp_dns_ttl }}"
 
 # ---------------------------------------------------------------------------
-# Play 4: Generate Agent ISO and deploy SNO (agent-based installer)
-#
-# Uses `openshift-install agent create image` — no SaaS API, no SSO required.
-# The pull secret is the only Red Hat credential needed.
-# Credentials (kubeconfig, kubeadmin-password) are generated locally under
-# sno_install_dir/auth/ by openshift-install itself.
+# Play 4: Generate Agent ISO and Deploy SNO
 # ---------------------------------------------------------------------------
 - name: Generate Agent ISO and Deploy SNO
   hosts: sno.openshift.toal.ca
   gather_facts: false
   connection: local
+  tags: sno_deploy_install
+
+  tasks:
+    - name: Install SNO
+      ansible.builtin.include_role:
+        name: sno_deploy
+        tasks_from: install.yml
+
+# ---------------------------------------------------------------------------
+# Play 5: Configure Keycloak OIDC client for OpenShift
+# ---------------------------------------------------------------------------
+- name: Configure Keycloak OIDC client for OpenShift
+  hosts: openshift
+  gather_facts: false
+  connection: local
+
+  tags: keycloak
 
   vars:
-    ocp_pull_secret: "{{ vault_ocp_pull_secret }}"
+    keycloak_context: ""
+    oidc_client_id: openshift
+    oidc_redirect_uri: "https://oauth-openshift.apps.{{ ocp_cluster_name }}.{{ ocp_base_domain }}/oauth2callback/{{ oidc_provider_name }}"
+    __oidc_keycloak_api_url: "{{ keycloak_url }}{{ keycloak_context }}"
 
-  tags: sno
+  module_defaults:
+    middleware_automation.keycloak.keycloak_realm:
+      auth_client_id: admin-cli
+      auth_keycloak_url: "{{ __oidc_keycloak_api_url }}"
+      auth_realm: master
+      auth_username: "{{ keycloak_admin_user }}"
+      auth_password: "{{ vault_keycloak_admin_password }}"
+      validate_certs: "{{ keycloak_validate_certs | default(true) }}"
+    middleware_automation.keycloak.keycloak_client:
+      auth_client_id: admin-cli
+      auth_keycloak_url: "{{ __oidc_keycloak_api_url }}"
+      auth_realm: master
+      auth_username: "{{ keycloak_admin_user }}"
+      auth_password: "{{ vault_keycloak_admin_password }}"
+      validate_certs: "{{ keycloak_validate_certs | default(true) }}"
 
   tasks:
 
-    # ------------------------------------------------------------------
-    # Step 0: Ensure sno_vm_id and sno_mac are populated.
-    # These are set as cacheable facts by the proxmox_sno_vm role, but
-    # in ephemeral EEs or when running --tags sno alone the cache is
-    # empty.  Re-query Proxmox whenever either value is missing.
-    # ------------------------------------------------------------------
-    - name: Retrieve VM info from Proxmox (needed when fact cache is empty)
-      community.proxmox.proxmox_vm_info:
-        api_host: "{{ hostvars['proxmox_api']['ansible_host'] }}"
-        api_user: "{{ proxmox_api_user }}"
-        api_port: "{{ hostvars['proxmox_api']['ansible_port'] }}"
-        api_token_id: "{{ proxmox_api_token_id }}"
-        api_token_secret: "{{ proxmox_api_token_secret }}"
-        validate_certs: "{{ proxmox_validate_certs }}"
-        node: "{{ proxmox_node }}"
-        name: "{{ sno_vm_name }}"
-        type: qemu
-        config: current
-      register: _sno_vm_info
-      when: (sno_vm_id | default('')) == '' or (sno_mac | default('')) == ''
-
-    - name: Set sno_vm_id and sno_mac from live Proxmox query
+    - name: Set OIDC client secret (use vault value or generate random)
       ansible.builtin.set_fact:
-        sno_vm_id: "{{ _sno_vm_info.proxmox_vms[0].vmid }}"
-        sno_mac: >-
-          {{ _sno_vm_info.proxmox_vms[0].config.net0
-             | regex_search('([0-9A-Fa-f]{2}(?::[0-9A-Fa-f]{2}){5})', '\1')
-             | first }}
-        cacheable: true
-      when: _sno_vm_info is not skipped
+        __oidc_client_secret: "{{ vault_oidc_client_secret | default(lookup('community.general.random_string', length=32, special=false)) }}"
+        __oidc_secret_generated: "{{ vault_oidc_client_secret is not defined }}"
+      no_log: true
 
-    - name: Ensure local install directories exist
-      ansible.builtin.file:
-        path: "{{ item }}"
-        state: directory
-        mode: "0750"
-      loop:
-        - "{{ sno_install_dir }}"
-        - "{{ sno_install_dir }}/auth"
+    - name: Ensure Keycloak realm exists
+      middleware_automation.keycloak.keycloak_realm:
+        realm: "{{ keycloak_realm }}"
+        id: "{{ keycloak_realm }}"
+        display_name: "{{ keycloak_realm_display_name | default(keycloak_realm | title) }}"
+        enabled: true
+        state: present
+      no_log: "{{ keycloak_no_log | default(true) }}"
 
-    # ------------------------------------------------------------------
-    # Step 1: Check whether a fresh ISO already exists on Proxmox
-    #         AND the local openshift-install state dir is intact.
-    #         If the state dir is missing (e.g. /tmp was cleared),
-    #         we must regenerate the ISO so wait-for has valid state.
-    # ------------------------------------------------------------------
-    - name: Check if ISO already exists on Proxmox and is less than 24 hours old
-      ansible.builtin.stat:
-        path: "{{ proxmox_iso_dir }}/{{ sno_iso_filename }}"
-        get_checksum: false
-      delegate_to: proxmox_host
-      register: proxmox_iso_stat
+    - name: Create OpenShift OIDC client in Keycloak
+      middleware_automation.keycloak.keycloak_client:
+        realm: "{{ keycloak_realm }}"
+        client_id: "{{ oidc_client_id }}"
+        name: "OpenShift - {{ ocp_cluster_name }}"
+        description: "OIDC client for OpenShift cluster {{ ocp_cluster_name }}.{{ ocp_base_domain }}"
+        enabled: true
+        protocol: openid-connect
+        public_client: false
+        standard_flow_enabled: true
+        implicit_flow_enabled: false
+        direct_access_grants_enabled: false
+        service_accounts_enabled: false
+        secret: "{{ __oidc_client_secret }}"
+        redirect_uris:
+          - "{{ oidc_redirect_uri }}"
+        web_origins:
+          - "+"
+        protocol_mappers:
+          - name: groups
+            protocol: openid-connect
+            protocolMapper: oidc-group-membership-mapper
+            config:
+              full.path: "false"
+              id.token.claim: "true"
+              access.token.claim: "true"
+              userinfo.token.claim: "true"
+              claim.name: groups
+        state: present
+      no_log: "{{ keycloak_no_log | default(true) }}"
 
-    - name: Check if local openshift-install state directory exists
-      ansible.builtin.stat:
-        path: "{{ sno_install_dir }}/.openshift_install_state"
-        get_checksum: false
-      register: install_state_stat
-
-    - name: Set fact - skip ISO build if recent ISO exists on Proxmox and local state is intact
-      ansible.builtin.set_fact:
-        sno_iso_fresh: >-
-          {{
-            proxmox_iso_stat.stat.exists and
-            (now(utc=true).timestamp() | int - proxmox_iso_stat.stat.mtime | int) < 86400 and
-            install_state_stat.stat.exists
-          }}
-
-    # ------------------------------------------------------------------
-    # Step 2: Get openshift-install binary
-    # Always ensure the binary is present — needed for both ISO generation
-    # and wait-for-install-complete regardless of sno_iso_fresh.
-    # Binaries are stored in sno_install_dir so they survive across runs
-    # when sno_install_dir is a mounted volume in an EE.
-    # ------------------------------------------------------------------
-    - name: Download openshift-install tarball
-      ansible.builtin.get_url:
-        url: "https://mirror.openshift.com/pub/openshift-v4/clients/ocp/stable-{{ ocp_version }}/openshift-install-linux.tar.gz"
-        dest: "{{ sno_install_dir }}/openshift-install-{{ ocp_version }}.tar.gz"
-        mode: "0644"
-        checksum: "{{ ocp_install_checksum | default(omit) }}"
-      register: ocp_install_tarball
-
-    - name: Extract openshift-install binary
-      ansible.builtin.unarchive:
-        src: "{{ sno_install_dir }}/openshift-install-{{ ocp_version }}.tar.gz"
-        dest: "{{ sno_install_dir }}"
-        remote_src: false
-        include:
-          - openshift-install
-      when: ocp_install_tarball.changed or not (sno_install_dir ~ '/openshift-install') is file
-
-    - name: Download openshift-client tarball
-      ansible.builtin.get_url:
-        url: "https://mirror.openshift.com/pub/openshift-v4/clients/ocp/stable-{{ ocp_version }}/openshift-client-linux.tar.gz"
-        dest: "{{ sno_install_dir }}/openshift-client-{{ ocp_version }}.tar.gz"
-        mode: "0644"
-        checksum: "{{ ocp_client_checksum | default(omit) }}"
-      register: ocp_client_tarball
-
-    - name: Extract oc binary
-      ansible.builtin.unarchive:
-        src: "{{ sno_install_dir }}/openshift-client-{{ ocp_version }}.tar.gz"
-        dest: "{{ sno_install_dir }}"
-        remote_src: false
-        include:
-          - oc
-      when: ocp_client_tarball.changed or not (sno_install_dir ~ '/oc') is file
-
-    # ------------------------------------------------------------------
-    # Step 3: Template agent installer config files (skipped if ISO is fresh)
-    # ------------------------------------------------------------------
-    - name: Template install-config.yaml
-      ansible.builtin.template:
-        src: templates/install-config.yaml.j2
-        dest: "{{ sno_install_dir }}/install-config.yaml"
-        mode: "0640"
-      when: not sno_iso_fresh
-
-    - name: Template agent-config.yaml
-      ansible.builtin.template:
-        src: templates/agent-config.yaml.j2
-        dest: "{{ sno_install_dir }}/agent-config.yaml"
-        mode: "0640"
-      when: not sno_iso_fresh
-
-    # ------------------------------------------------------------------
-    # Step 4: Generate discovery ISO (skipped if ISO is fresh)
-    # Note: openshift-install consumes (moves) the config files into
-    # openshift-install-state/ — this is expected behaviour.
-    # ------------------------------------------------------------------
-    - name: Generate agent-based installer ISO
-      ansible.builtin.command:
-        cmd: "{{ sno_install_dir }}/openshift-install agent create image --dir {{ sno_install_dir }}"
-      when: not sno_iso_fresh
-
-    # ------------------------------------------------------------------
-    # Step 5: Upload ISO to Proxmox and attach to VM
-    # ------------------------------------------------------------------
-    - name: Copy discovery ISO to Proxmox ISO storage
-      ansible.builtin.copy:
-        src: "{{ sno_install_dir }}/{{ sno_iso_filename }}"
-        dest: "{{ proxmox_iso_dir }}/{{ sno_iso_filename }}"
-        mode: "0644"
-      delegate_to: proxmox_host
-      when: not sno_iso_fresh
-
-    - name: Attach ISO to VM as CDROM
-      ansible.builtin.command:
-        cmd: "qm set {{ sno_vm_id }} --ide2 {{ proxmox_iso_storage }}:iso/{{ sno_iso_filename }},media=cdrom"
-      delegate_to: proxmox_host
-      changed_when: true
-
-    - name: Ensure boot order prefers disk, falls back to CDROM
-      # order=scsi0;ide2: OVMF tries scsi0 first; on first boot the disk has
-      # no EFI application so OVMF falls through to ide2 (the agent ISO).
-      # After RHCOS writes its EFI entry to the disk, subsequent reboots boot
-      # directly from scsi0 — the CDROM is never tried again, breaking the loop.
-      ansible.builtin.command:
-        cmd: "qm set {{ sno_vm_id }} --boot order=scsi0;ide2"
-      delegate_to: proxmox_host
-      changed_when: true
-
-    # ------------------------------------------------------------------
-    # Step 6: Boot the VM
-    # ------------------------------------------------------------------
-    - name: Start SNO VM
-      community.proxmox.proxmox_kvm:
-        api_host: "{{ hostvars['proxmox_api']['ansible_host'] }}"
-        api_user: "{{ proxmox_api_user }}"
-        api_port: "{{ hostvars['proxmox_api']['ansible_port'] }}"
-        api_token_id: "{{ proxmox_api_token_id }}"
-        api_token_secret: "{{ proxmox_api_token_secret }}"
-        validate_certs: "{{ proxmox_validate_certs }}"
-        node: "{{ proxmox_node }}"
-        name: "{{ sno_vm_name }}"
-        state: started
-
-    # ------------------------------------------------------------------
-    # Step 7: Persist credentials to Proxmox host
-    # The EE is ephemeral — copy auth files to a durable location before
-    # the container exits.  sno_credentials_dir defaults to
-    # /root/sno-<cluster_name> on proxmox_host.
-    # ------------------------------------------------------------------
-    - name: Create credentials directory on Proxmox host
-      ansible.builtin.file:
-        path: "{{ sno_credentials_dir }}"
-        state: directory
-        mode: "0700"
-      delegate_to: proxmox_host
-
-    - name: Copy kubeconfig to Proxmox host
-      ansible.builtin.copy:
-        src: "{{ sno_install_dir }}/auth/kubeconfig"
-        dest: "{{ sno_credentials_dir }}/kubeconfig"
-        mode: "0600"
-      delegate_to: proxmox_host
-
-    - name: Copy kubeadmin-password to Proxmox host
-      ansible.builtin.copy:
-        src: "{{ sno_install_dir }}/auth/kubeadmin-password"
-        dest: "{{ sno_credentials_dir }}/kubeadmin-password"
-        mode: "0600"
-      delegate_to: proxmox_host
-    # ------------------------------------------------------------------
-    # Step 8: Wait for installation to complete (~60-90 min)
-    # Credentials land in sno_install_dir/auth/ automatically.
-    # Inline poll (poll: 30) is used rather than fire-and-forget async
-    # because the connection is local — no SSH timeout risk — and the
-    # poll: 0 + async_status pattern stores job state in ~/.ansible_async
-    # inside the EE container, which is lost if the EE is restarted.
-    # Ensure your job/EE timeout is set to at least 6000 s (100 min).
-    # ------------------------------------------------------------------
-    - name: Wait for SNO installation to complete
-      ansible.builtin.command:
-        cmd: "{{ sno_install_dir }}/openshift-install agent wait-for install-complete --dir {{ sno_install_dir }} --log-level=info"
-      async: 5400
-      poll: 30
-
-    # ------------------------------------------------------------------
-    # Step 9: Eject CDROM so the VM never boots the agent ISO again
-    # ------------------------------------------------------------------
-    - name: Eject CDROM after successful installation
-      ansible.builtin.command:
-        cmd: "qm set {{ sno_vm_id }} --ide2 none,media=cdrom"
-      delegate_to: proxmox_host
-      changed_when: true
-
-
-    - name: Display post-install info
+    - name: Display generated client secret (save this to vault!)
       ansible.builtin.debug:
         msg:
-          - "SNO installation complete!"
-          - "API URL        : https://api.{{ ocp_cluster_name }}.{{ ocp_base_domain }}:6443"
-          - "Console        : https://console-openshift-console.apps.{{ ocp_cluster_name }}.{{ ocp_base_domain }}"
-          - "Kubeconfig     : {{ sno_credentials_dir }}/kubeconfig  (on proxmox_host)"
-          - "kubeadmin pass : {{ sno_credentials_dir }}/kubeadmin-password  (on proxmox_host)"
+          - "*** GENERATED OIDC CLIENT SECRET — SAVE THIS TO VAULT ***"
+          - "vault_oidc_client_secret: {{ __oidc_client_secret }}"
+          - ""
+          - "Set this in host_vars or pass as --extra-vars on future runs."
+      when: __oidc_secret_generated | bool
+
+    - name: Display Keycloak configuration summary
+      ansible.builtin.debug:
+        msg:
+          - "Keycloak OIDC client configured:"
+          - "  Realm   : {{ keycloak_realm }}"
+          - "  Client  : {{ oidc_client_id }}"
+          - "  Issuer  : {{ keycloak_url }}{{ keycloak_context }}/realms/{{ keycloak_realm }}"
+          - "  Redirect: {{ oidc_redirect_uri }}"
+        verbosity: 1
+
+# ---------------------------------------------------------------------------
+# Play 6: Post-install OpenShift configuration
+# Configure OIDC OAuth, cert-manager, and delete kubeadmin
+# ---------------------------------------------------------------------------
+- name: Configure OpenShift post-install
+  hosts: sno.openshift.toal.ca
+  gather_facts: false
+  connection: local
+
+  environment:
+    KUBECONFIG: "{{ ocp_kubeconfig | default('~/.kube/config') }}"
+    K8S_AUTH_VERIFY_SSL: "false"
+
+  tags:
+    - sno_deploy_oidc
+    - sno_deploy_certmanager
+    - sno_deploy_delete_kubeadmin
+
+  tasks:
+    - name: Configure OpenShift OAuth with OIDC
+      ansible.builtin.include_role:
+        name: sno_deploy
+        tasks_from: configure_oidc.yml
+      tags: sno_deploy_oidc
+
+    - name: Configure cert-manager and LetsEncrypt certificates
+      ansible.builtin.include_role:
+        name: sno_deploy
+        tasks_from: configure_certmanager.yml
+      tags: sno_deploy_certmanager
+
+    - name: Delete kubeadmin user
+      ansible.builtin.include_role:
+        name: sno_deploy
+        tasks_from: delete_kubeadmin.yml
+      tags:
+        - never
+        - sno_deploy_delete_kubeadmin
+
+# ---------------------------------------------------------------------------
+# Play 7: Install LVM Storage for persistent volumes
+# ---------------------------------------------------------------------------
+- name: Configure LVM Storage for persistent volumes
+  hosts: sno.openshift.toal.ca
+  gather_facts: false
+  connection: local
+  tags: sno_deploy_lvms
+
+  environment:
+    KUBECONFIG: "{{ ocp_kubeconfig | default('~/.kube/config') }}"
+    K8S_AUTH_VERIFY_SSL: "false"
+
+  roles:
+    - role: lvms_operator
+
+# ---------------------------------------------------------------------------
+# Play 8: Deploy NFS provisioner for ReadWriteMany storage
+# Set nfs_provisioner_external_server / nfs_provisioner_external_path to use
+# a pre-existing NFS share (e.g. 192.168.129.100:/mnt/BIGPOOL/NoBackups/OCPNFS).
+# When those are unset, an in-cluster NFS server is deployed; LVMS (Play 7) must
+# have run first to provide the backing RWO PVC.
+# ---------------------------------------------------------------------------
+- name: Deploy in-cluster NFS provisioner
+  hosts: sno.openshift.toal.ca
+  gather_facts: false
+  connection: local
+  tags: sno_deploy_nfs
+
+  environment:
+    KUBECONFIG: "{{ ocp_kubeconfig | default('~/.kube/config') }}"
+    K8S_AUTH_VERIFY_SSL: "false"
+
+  roles:
+    - role: nfs_provisioner
+
+# ---------------------------------------------------------------------------
+# Play 9: Provision ServiceAccounts for application deployers
+# ---------------------------------------------------------------------------
+- name: Provision OpenShift service accounts
+  hosts: sno.openshift.toal.ca
+  gather_facts: false
+  connection: local
+
+  environment:
+    KUBECONFIG: "{{ ocp_kubeconfig | default('~/.kube/config') }}"
+    K8S_AUTH_VERIFY_SSL: "false"
+
+  tags:
+    - never
+    - sno_deploy_service_accounts
+
+  roles:
+    - role: ocp_service_account
+      ocp_service_account_name: aap-deployer
+      ocp_service_account_namespace: aap
+      ocp_service_account_cluster_role_rules:
+        - apiGroups: [""]
+          resources: ["namespaces"]
+          verbs: ["get", "list", "create", "patch"]
+        - apiGroups: [""]
+          resources: ["secrets"]
+          verbs: ["get", "list", "watch", "create", "patch"]
+        - apiGroups: [""]
+          resources: ["serviceaccounts"]
+          verbs: ["get", "list", "watch"]
+        - apiGroups: ["apps"]
+          resources: ["deployments"]
+          verbs: ["get", "list", "watch"]
+        - apiGroups: ["operators.coreos.com"]
+          resources: ["operatorgroups", "subscriptions", "clusterserviceversions"]
+          verbs: ["get", "list", "create", "patch", "watch"]
+        - apiGroups: ["apiextensions.k8s.io"]
+          resources: ["customresourcedefinitions"]
+          verbs: ["get", "list", "watch"]
+        - apiGroups: ["aap.ansible.com"]
+          resources: ["ansibleautomationplatforms"]
+          verbs: ["get", "list", "create", "patch", "watch"]

playbooks/satellite.yml

@@ -272,12 +272,6 @@
 
 #TODO Automatically set up DNS GSSAPI per:  https://access.redhat.com/documentation/en-us/red_hat_satellite/6.8/html/installing_satellite_server_from_a_connected_network/configuring-external-services#configuring-external-idm-dns_satellite
 
-- name: Set up Basic Lab Packages
-  hosts: "{{ vm_name }}"
-  become: yes
-  roles:
-    - role: toal-common
-
 - name: Install Satellite Servers
   hosts: "{{ vm_name }}"
   become: true

playbooks/site.yml

@@ -6,7 +6,6 @@
 
    - name: linux-system-roles.network
      when: network_connections is defined
-   - name: toal-common
 
 - name: Set Network OS from Netbox info.
   gather_facts: no

roles/aap_operator/defaults/main.yml

@@ -0,0 +1,29 @@
+---
+# --- OLM subscription ---
+aap_operator_namespace: aap
+aap_operator_channel: "stable-2.6"
+aap_operator_source: redhat-operators
+aap_operator_name: ansible-automation-platform-operator
+aap_operator_wait_timeout: 1800
+
+# --- AnsibleAutomationPlatform CR ---
+aap_operator_platform_name: aap
+
+# --- Components (set disabled: true to skip) ---
+aap_operator_controller_disabled: false
+aap_operator_hub_disabled: false
+aap_operator_eda_disabled: false
+
+# --- Storage ---
+# RWO StorageClass for PostgreSQL (all components)
+aap_operator_storage_class: lvms-vg-data
+# RWX StorageClass for Hub file/artifact storage
+aap_operator_hub_file_storage_class: nfs-client
+aap_operator_hub_file_storage_size: 10Gi
+
+# --- Admin ---
+aap_operator_admin_user: admin
+
+# --- Routing (optional) ---
+# Set to a custom hostname to override the auto-generated Controller route
+# aap_operator_controller_route_host: aap.example.com

roles/aap_operator/meta/argument_specs.yml

@@ -0,0 +1,67 @@
+---
+argument_specs:
+  main:
+    short_description: Install AAP via OpenShift OLM operator (AnsibleAutomationPlatform CR)
+    description:
+      - Installs the Ansible Automation Platform operator via OLM and creates a
+        single AnsibleAutomationPlatform CR that manages Controller, Hub, and EDA.
+    options:
+      aap_operator_namespace:
+        description: Namespace for the AAP operator and platform instance.
+        type: str
+        default: aap
+      aap_operator_channel:
+        description: OLM subscription channel.
+        type: str
+        default: "stable-2.6"
+      aap_operator_source:
+        description: OLM catalog source name.
+        type: str
+        default: redhat-operators
+      aap_operator_name:
+        description: Operator package name in the catalog.
+        type: str
+        default: ansible-automation-platform-operator
+      aap_operator_wait_timeout:
+        description: Seconds to wait for operator and platform to become ready.
+        type: int
+        default: 1800
+      aap_operator_platform_name:
+        description: Name of the AnsibleAutomationPlatform CR.
+        type: str
+        default: aap
+      aap_operator_controller_disabled:
+        description: Set true to skip deploying Automation Controller.
+        type: bool
+        default: false
+      aap_operator_hub_disabled:
+        description: Set true to skip deploying Automation Hub.
+        type: bool
+        default: false
+      aap_operator_eda_disabled:
+        description: Set true to skip deploying Event-Driven Ansible.
+        type: bool
+        default: false
+      aap_operator_storage_class:
+        description: StorageClass for PostgreSQL persistent volumes (RWO).
+        type: str
+        default: lvms-vg-data
+      aap_operator_hub_file_storage_class:
+        description: StorageClass for Hub file/artifact storage (RWX).
+        type: str
+        default: nfs-client
+      aap_operator_hub_file_storage_size:
+        description: Size of the Hub file storage PVC.
+        type: str
+        default: 10Gi
+      aap_operator_admin_user:
+        description: Admin username for the platform.
+        type: str
+        default: admin
+      aap_operator_controller_route_host:
+        description: >
+          Custom hostname for the Automation Controller Route.
+          When set, overrides the auto-generated route hostname (e.g. aap.example.com).
+          Leave unset to use the default apps subdomain route.
+        type: str
+        required: false

roles/aap_operator/meta/main.yml

@@ -0,0 +1,18 @@
+---
+galaxy_info:
+  author: ptoal
+  description: Install Ansible Automation Platform via OpenShift OLM operator
+  license: MIT
+  min_ansible_version: "2.16"
+  platforms:
+    - name: GenericLinux
+      versions:
+        - all
+  galaxy_tags:
+    - openshift
+    - aap
+    - operator
+    - olm
+    - ansible
+
+dependencies: []

roles/aap_operator/tasks/configure_oidc.yml

@@ -0,0 +1,4 @@
+---
+# OIDC is configured via the AAP Gateway API, not via this role.
+# See: playbooks/deploy_aap.yml --tags aap_configure_keycloak,aap_configure_oidc
+# Uses: infra.aap_configuration.gateway_authenticators

roles/aap_operator/tasks/main.yml

@@ -0,0 +1,166 @@
+---
+# Install Ansible Automation Platform via OpenShift OLM operator.
+#
+# Deploys the AAP operator, then creates a single AnsibleAutomationPlatform
+# CR that manages Controller, Hub, and EDA as a unified platform.
+# All tasks are idempotent (kubernetes.core.k8s state: present).
+
+# ------------------------------------------------------------------
+# Step 1: Install AAP operator via OLM
+# ------------------------------------------------------------------
+- name: Create AAP namespace
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: v1
+      kind: Namespace
+      metadata:
+        name: "{{ aap_operator_namespace }}"
+
+- name: Read global pull secret
+  kubernetes.core.k8s_info:
+    api_version: v1
+    kind: Secret
+    namespace: openshift-config
+    name: pull-secret
+  register: __aap_operator_global_pull_secret
+
+- name: Copy pull secret to AAP namespace
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: v1
+      kind: Secret
+      metadata:
+        name: redhat-operators-pull-secret
+        namespace: "{{ aap_operator_namespace }}"
+      type: kubernetes.io/dockerconfigjson
+      data:
+        .dockerconfigjson: "{{ __aap_operator_global_pull_secret.resources[0].data['.dockerconfigjson'] }}"
+  no_log: false
+
+- name: Create OperatorGroup for AAP
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: operators.coreos.com/v1
+      kind: OperatorGroup
+      metadata:
+        name: "{{ aap_operator_name }}"
+        namespace: "{{ aap_operator_namespace }}"
+      spec:
+        targetNamespaces:
+          - "{{ aap_operator_namespace }}"
+        upgradeStrategy: Default
+
+- name: Subscribe to AAP operator
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: operators.coreos.com/v1alpha1
+      kind: Subscription
+      metadata:
+        name: "{{ aap_operator_name }}"
+        namespace: "{{ aap_operator_namespace }}"
+      spec:
+        channel: "{{ aap_operator_channel }}"
+        installPlanApproval: Automatic
+        name: "{{ aap_operator_name }}"
+        source: "{{ aap_operator_source }}"
+        sourceNamespace: openshift-marketplace
+
+# ------------------------------------------------------------------
+# Step 2: Wait for operator to be ready
+# ------------------------------------------------------------------
+- name: Wait for AnsibleAutomationPlatform CRD to be available
+  kubernetes.core.k8s_info:
+    api_version: apiextensions.k8s.io/v1
+    kind: CustomResourceDefinition
+    name: ansibleautomationplatforms.aap.ansible.com
+  register: __aap_operator_crd
+  until: __aap_operator_crd.resources | length > 0
+  retries: "{{ __aap_operator_wait_retries }}"
+  delay: 10
+
+- name: Wait for AAP operator deployments to be ready
+  kubernetes.core.k8s_info:
+    api_version: apps/v1
+    kind: Deployment
+    namespace: "{{ aap_operator_namespace }}"
+    label_selectors:
+      - "operators.coreos.com/{{ aap_operator_name }}.{{ aap_operator_namespace }}"
+  register: __aap_operator_deploy
+  until: >-
+    __aap_operator_deploy.resources | length > 0 and
+    (__aap_operator_deploy.resources
+     | rejectattr('status.readyReplicas', 'undefined')
+     | selectattr('status.readyReplicas', '>=', 1)
+     | list | length) == (__aap_operator_deploy.resources | length)
+  retries: "{{ __aap_operator_wait_retries }}"
+  delay: 10
+
+# ------------------------------------------------------------------
+# Step 3: Deploy the unified AnsibleAutomationPlatform
+# ------------------------------------------------------------------
+- name: Create AnsibleAutomationPlatform
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: aap.ansible.com/v1alpha1
+      kind: AnsibleAutomationPlatform
+      metadata:
+        name: "{{ aap_operator_platform_name }}"
+        namespace: "{{ aap_operator_namespace }}"
+      spec:
+        admin_user: "{{ aap_operator_admin_user }}"
+        # PostgreSQL storage for all components (RWO)
+        database:
+          postgres_storage_class: "{{ aap_operator_storage_class }}"
+        # Component toggles and per-component config
+        controller:
+          disabled: "{{ aap_operator_controller_disabled | bool }}"
+          route_host: "{{ aap_operator_controller_route_host | default(omit) }}"
+        hub:
+          disabled: "{{ aap_operator_hub_disabled | bool }}"
+          # Hub file/artifact storage (RWX) — must be under hub:
+          storage_type: file
+          file_storage_storage_class: "{{ aap_operator_hub_file_storage_class }}"
+          file_storage_size: "{{ aap_operator_hub_file_storage_size }}"
+        eda:
+          disabled: "{{ aap_operator_eda_disabled | bool }}"
+
+# ------------------------------------------------------------------
+# Step 4: Wait for platform to be ready
+# ------------------------------------------------------------------
+- name: Wait for AnsibleAutomationPlatform to be ready
+  kubernetes.core.k8s_info:
+    api_version: aap.ansible.com/v1alpha1
+    kind: AnsibleAutomationPlatform
+    namespace: "{{ aap_operator_namespace }}"
+    name: "{{ aap_operator_platform_name }}"
+  register: __aap_operator_platform_status
+  ignore_errors: true
+  until: >-
+    __aap_operator_platform_status.resources is defined and
+    __aap_operator_platform_status.resources | length > 0 and
+    (__aap_operator_platform_status.resources[0].status.conditions | default([])
+     | selectattr('type', '==', 'Running')
+     | selectattr('status', '==', 'True') | list | length > 0)
+  retries: "{{ __aap_operator_wait_retries }}"
+  delay: 10
+
+# ------------------------------------------------------------------
+# Step 5: Display summary
+# ------------------------------------------------------------------
+- name: Display AAP deployment summary
+  ansible.builtin.debug:
+    msg:
+      - "Ansible Automation Platform deployment complete!"
+      - "  Namespace  : {{ aap_operator_namespace }}"
+      - "  Platform CR: {{ aap_operator_platform_name }}"
+      - "  Controller : {{ 'disabled' if aap_operator_controller_disabled else 'enabled' }}"
+      - "  Hub        : {{ 'disabled' if aap_operator_hub_disabled else 'enabled' }}"
+      - "  EDA        : {{ 'disabled' if aap_operator_eda_disabled else 'enabled' }}"
+      - ""
+      - "Admin password secret: {{ aap_operator_platform_name }}-admin-password"
+      - "Retrieve with: oc get secret {{ aap_operator_platform_name }}-admin-password -n {{ aap_operator_namespace }} -o jsonpath='{.data.password}' | base64 -d"

roles/aap_operator/vars/main.yml

@@ -0,0 +1,3 @@
+---
+# Computed internal variables - do not override
+__aap_operator_wait_retries: "{{ (aap_operator_wait_timeout / 10) | int }}"

roles/dnsmadeeasy_record/README.md

@@ -0,0 +1,58 @@
+# dnsmadeeasy_record
+
+Manages DNS records in DNS Made Easy via the `community.general.dnsmadeeasy` module.
+
+Accepts a list of record entries and creates or updates each one.
+
+## Requirements
+
+- `community.general` collection
+- DNS Made Easy account credentials
+
+## Role Variables
+
+| Variable | Default | Description |
+|---|---|---|
+| `dnsmadeeasy_record_account_key` | *required* | DNS Made Easy account key |
+| `dnsmadeeasy_record_account_secret` | *required* | DNS Made Easy account secret (sensitive) |
+| `dnsmadeeasy_record_entries` | `[]` | List of DNS record entries (see below) |
+
+### Entry format
+
+Each entry in `dnsmadeeasy_record_entries` requires:
+
+| Field | Required | Default | Description |
+|---|---|---|---|
+| `domain` | yes | | DNS zone (e.g. `openshift.toal.ca`) |
+| `record_name` | yes | | Record name within the zone |
+| `record_type` | yes | | DNS record type (A, CNAME, etc.) |
+| `record_value` | yes | | Target value |
+| `record_ttl` | no | `1800` | TTL in seconds |
+
+## Example Playbook
+
+```yaml
+- name: Configure public DNS records
+  hosts: sno.openshift.toal.ca
+  gather_facts: false
+  connection: local
+
+  roles:
+    - role: dnsmadeeasy_record
+      dnsmadeeasy_record_account_key: "{{ dme_account_key }}"
+      dnsmadeeasy_record_account_secret: "{{ dme_account_secret }}"
+      dnsmadeeasy_record_entries:
+        - domain: openshift.toal.ca
+          record_name: api.sno
+          record_type: A
+          record_value: 203.0.113.1
+          record_ttl: 300
+```
+
+## License
+
+MIT
+
+## Author
+
+ptoal

roles/dnsmadeeasy_record/defaults/main.yml

@@ -0,0 +1,24 @@
+---
+# DNS Made Easy API credentials
+# dnsmadeeasy_record_account_key: ""    # required
+# dnsmadeeasy_record_account_secret: "" # required (sensitive)
+
+# List of DNS records to create/update.
+#
+# Each entry requires:
+#   domain:       DNS zone (e.g. "openshift.toal.ca")
+#   record_name:  record name within the zone (e.g. "api.sno")
+#   record_type:  DNS record type (A, CNAME, etc.)
+#   record_value: target value (IP address or hostname)
+#
+# Optional per entry:
+#   record_ttl:   TTL in seconds (default: 1800)
+#
+# Example:
+#   dnsmadeeasy_record_entries:
+#     - domain: openshift.toal.ca
+#       record_name: api.sno
+#       record_type: A
+#       record_value: 203.0.113.1
+#       record_ttl: 300
+dnsmadeeasy_record_entries: []

roles/dnsmadeeasy_record/meta/argument_specs.yml

@@ -0,0 +1,24 @@
+---
+argument_specs:
+  main:
+    short_description: Manage DNS records in DNS Made Easy
+    description:
+      - Creates or updates DNS records via the DNS Made Easy API
+        using the community.general.dnsmadeeasy module.
+    options:
+      dnsmadeeasy_record_account_key:
+        description: DNS Made Easy account key.
+        type: str
+        required: true
+      dnsmadeeasy_record_account_secret:
+        description: DNS Made Easy account secret.
+        type: str
+        required: true
+        no_log: true
+      dnsmadeeasy_record_entries:
+        description: >-
+          List of DNS record entries. Each entry requires C(domain), C(record_name),
+          C(record_type), and C(record_value). Optional C(record_ttl) defaults to 1800.
+        type: list
+        elements: dict
+        default: []

roles/dnsmadeeasy_record/meta/main.yml

@@ -0,0 +1,15 @@
+---
+galaxy_info:
+  author: ptoal
+  description: Manage DNS records in DNS Made Easy
+  license: MIT
+  min_ansible_version: "2.16"
+  platforms:
+    - name: GenericLinux
+      versions:
+        - all
+  galaxy_tags:
+    - dns
+    - dnsmadeeasy
+
+dependencies: []

roles/dnsmadeeasy_record/tasks/main.yml

@@ -0,0 +1,14 @@
+---
+- name: Manage DNS Made Easy records
+  community.general.dnsmadeeasy:
+    account_key: "{{ dnsmadeeasy_record_account_key }}"
+    account_secret: "{{ dnsmadeeasy_record_account_secret }}"
+    domain: "{{ item.domain }}"
+    record_name: "{{ item.record_name }}"
+    record_type: "{{ item.record_type }}"
+    record_value: "{{ item.record_value }}"
+    record_ttl: "{{ item.record_ttl | default(1800) }}"
+    state: present
+  loop: "{{ dnsmadeeasy_record_entries }}"
+  loop_control:
+    label: "{{ item.record_name }}.{{ item.domain }} ({{ item.record_type }})"

roles/lvms_operator/defaults/main.yml

@@ -0,0 +1,13 @@
+---
+# --- OLM subscription ---
+lvms_operator_namespace: openshift-storage
+lvms_operator_channel: "stable-4.21"
+lvms_operator_source: redhat-operators
+lvms_operator_name: lvms-operator
+lvms_operator_wait_timeout: 300
+
+# --- LVMCluster ---
+lvms_operator_vg_name: vg-data
+lvms_operator_device_paths:
+  - /dev/sdb
+lvms_operator_storage_class_name: lvms-vg-data

roles/lvms_operator/meta/argument_specs.yml

@@ -0,0 +1,42 @@
+---
+argument_specs:
+  main:
+    short_description: Install LVMS operator for persistent storage on OpenShift
+    description:
+      - Installs the LVM Storage operator via OLM and creates an LVMCluster
+        with a volume group backed by specified block devices.
+    options:
+      lvms_operator_namespace:
+        description: Namespace for the LVMS operator.
+        type: str
+        default: openshift-storage
+      lvms_operator_channel:
+        description: OLM subscription channel.
+        type: str
+        default: "stable-4.21"
+      lvms_operator_source:
+        description: OLM catalog source name.
+        type: str
+        default: redhat-operators
+      lvms_operator_name:
+        description: Operator package name in the catalog.
+        type: str
+        default: lvms-operator
+      lvms_operator_wait_timeout:
+        description: Seconds to wait for operator and LVMCluster to become ready.
+        type: int
+        default: 300
+      lvms_operator_vg_name:
+        description: Name of the volume group to create in the LVMCluster.
+        type: str
+        default: vg-data
+      lvms_operator_device_paths:
+        description: List of block device paths to include in the volume group.
+        type: list
+        elements: str
+        default:
+          - /dev/sdb
+      lvms_operator_storage_class_name:
+        description: Name of the StorageClass created by LVMS for this volume group.
+        type: str
+        default: lvms-vg-data

roles/lvms_operator/meta/main.yml

@@ -0,0 +1,18 @@
+---
+galaxy_info:
+  author: ptoal
+  description: Install LVM Storage (LVMS) operator on OpenShift for persistent volumes
+  license: MIT
+  min_ansible_version: "2.16"
+  platforms:
+    - name: GenericLinux
+      versions:
+        - all
+  galaxy_tags:
+    - openshift
+    - lvms
+    - storage
+    - operator
+    - olm
+
+dependencies: []

roles/lvms_operator/tasks/main.yml

@@ -0,0 +1,135 @@
+---
+# Install LVM Storage (LVMS) operator via OpenShift OLM.
+#
+# Creates an LVMCluster with a volume group backed by the specified
+# block devices, providing a StorageClass for persistent volume claims.
+# All tasks are idempotent (kubernetes.core.k8s state: present).
+
+# ------------------------------------------------------------------
+# Step 1: Install LVMS operator via OLM
+# ------------------------------------------------------------------
+- name: Create LVMS namespace
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: v1
+      kind: Namespace
+      metadata:
+        name: "{{ lvms_operator_namespace }}"
+
+- name: Create OperatorGroup for LVMS
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: operators.coreos.com/v1
+      kind: OperatorGroup
+      metadata:
+        name: "{{ lvms_operator_name }}"
+        namespace: "{{ lvms_operator_namespace }}"
+      spec:
+        targetNamespaces:
+          - "{{ lvms_operator_namespace }}"
+        upgradeStrategy: Default
+
+- name: Subscribe to LVMS operator
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: operators.coreos.com/v1alpha1
+      kind: Subscription
+      metadata:
+        name: "{{ lvms_operator_name }}"
+        namespace: "{{ lvms_operator_namespace }}"
+      spec:
+        channel: "{{ lvms_operator_channel }}"
+        installPlanApproval: Automatic
+        name: "{{ lvms_operator_name }}"
+        source: "{{ lvms_operator_source }}"
+        sourceNamespace: openshift-marketplace
+
+# ------------------------------------------------------------------
+# Step 2: Wait for operator to be ready
+# ------------------------------------------------------------------
+- name: Wait for LVMCluster CRD to be available
+  kubernetes.core.k8s_info:
+    api_version: apiextensions.k8s.io/v1
+    kind: CustomResourceDefinition
+    name: lvmclusters.lvm.topolvm.io
+  register: __lvms_operator_crd
+  until: __lvms_operator_crd.resources | length > 0
+  retries: "{{ __lvms_operator_wait_retries }}"
+  delay: 10
+
+- name: Wait for LVMS operator deployment to be ready
+  kubernetes.core.k8s_info:
+    api_version: apps/v1
+    kind: Deployment
+    namespace: "{{ lvms_operator_namespace }}"
+    label_selectors:
+      - "operators.coreos.com/{{ lvms_operator_name }}.{{ lvms_operator_namespace }}"
+  register: __lvms_operator_deploy
+  until: >-
+    __lvms_operator_deploy.resources | length > 0 and
+    (__lvms_operator_deploy.resources
+     | rejectattr('status.readyReplicas', 'undefined')
+     | selectattr('status.readyReplicas', '>=', 1)
+     | list | length) == (__lvms_operator_deploy.resources | length)
+  retries: "{{ __lvms_operator_wait_retries }}"
+  delay: 10
+
+# ------------------------------------------------------------------
+# Step 3: Create LVMCluster
+# ------------------------------------------------------------------
+- name: Create LVMCluster
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: lvm.topolvm.io/v1alpha1
+      kind: LVMCluster
+      metadata:
+        name: lvms-cluster
+        namespace: "{{ lvms_operator_namespace }}"
+      spec:
+        storage:
+          deviceClasses:
+            - name: "{{ lvms_operator_vg_name }}"
+              default: true
+              deviceSelector:
+                paths: "{{ lvms_operator_device_paths }}"
+              thinPoolConfig:
+                name: thin-pool
+                sizePercent: 90
+                overprovisionRatio: 10
+
+# ------------------------------------------------------------------
+# Step 4: Wait for LVMCluster to be ready
+# ------------------------------------------------------------------
+- name: Wait for LVMCluster to be ready
+  kubernetes.core.k8s_info:
+    api_version: lvm.topolvm.io/v1alpha1
+    kind: LVMCluster
+    namespace: "{{ lvms_operator_namespace }}"
+    name: lvms-cluster
+  register: __lvms_operator_cluster_status
+  until: >-
+    __lvms_operator_cluster_status.resources | length > 0 and
+    (__lvms_operator_cluster_status.resources[0].status.state | default('')) == 'Ready'
+  retries: "{{ __lvms_operator_wait_retries }}"
+  delay: 10
+
+- name: Verify StorageClass exists
+  kubernetes.core.k8s_info:
+    api_version: storage.k8s.io/v1
+    kind: StorageClass
+    name: "{{ lvms_operator_storage_class_name }}"
+  register: __lvms_operator_sc
+  failed_when: __lvms_operator_sc.resources | length == 0
+
+- name: Display LVMS summary
+  ansible.builtin.debug:
+    msg:
+      - "LVM Storage deployment complete!"
+      - "  Namespace     : {{ lvms_operator_namespace }}"
+      - "  Volume Group  : {{ lvms_operator_vg_name }}"
+      - "  Device Paths  : {{ lvms_operator_device_paths | join(', ') }}"
+      - "  StorageClass  : {{ lvms_operator_storage_class_name }}"

roles/lvms_operator/vars/main.yml

@@ -0,0 +1,3 @@
+---
+# Computed internal variables - do not override
+__lvms_operator_wait_retries: "{{ (lvms_operator_wait_timeout / 10) | int }}"

roles/nfs_provisioner/defaults/main.yml

@@ -0,0 +1,23 @@
+---
+# --- Namespace ---
+nfs_provisioner_namespace: nfs-provisioner
+
+# --- External NFS server (set these to use a pre-existing NFS share) ---
+# When nfs_provisioner_external_server is set, the in-cluster NFS server is
+# not deployed; the provisioner points directly at the external share.
+nfs_provisioner_external_server: ""   # e.g. 192.168.129.100
+nfs_provisioner_external_path: ""     # e.g. /mnt/BIGPOOL/NoBackups/OCPNFS
+
+# --- Backing storage for in-cluster NFS server (ignored when external_server is set) ---
+nfs_provisioner_storage_class: lvms-vg-data
+nfs_provisioner_storage_size: 50Gi
+nfs_provisioner_server_image: registry.k8s.io/volume-nfs:0.8
+nfs_provisioner_export_path: /exports
+
+# --- NFS provisioner ---
+nfs_provisioner_name: nfs-client
+nfs_provisioner_storage_class_name: nfs-client
+nfs_provisioner_image: registry.k8s.io/sig-storage/nfs-subdir-external-provisioner:v4.0.2
+
+# --- Wait ---
+nfs_provisioner_wait_timeout: 300

roles/nfs_provisioner/meta/argument_specs.yml

@@ -0,0 +1,67 @@
+---
+argument_specs:
+  main:
+    short_description: Deploy NFS provisioner (external or in-cluster) for RWX storage on OpenShift
+    description:
+      - Deploys the nfs-subdir-external-provisioner and a ReadWriteMany StorageClass.
+      - When nfs_provisioner_external_server is set, points directly at a pre-existing
+        NFS share (no in-cluster NFS server pod is deployed).
+      - When nfs_provisioner_external_server is empty, deploys an in-cluster NFS server
+        pod backed by an LVMS PVC.
+    options:
+      nfs_provisioner_namespace:
+        description: Namespace for the NFS provisioner (and optional in-cluster NFS server).
+        type: str
+        default: nfs-provisioner
+      nfs_provisioner_external_server:
+        description: >-
+          IP or hostname of a pre-existing external NFS server. When set, the
+          in-cluster NFS server pod is not deployed. Leave empty to use in-cluster mode.
+        type: str
+        default: ""
+      nfs_provisioner_external_path:
+        description: >-
+          Exported path on the external NFS server.
+          Required when nfs_provisioner_external_server is set.
+        type: str
+        default: ""
+      nfs_provisioner_storage_class:
+        description: >-
+          StorageClass (RWO) for the in-cluster NFS server backing PVC.
+          Ignored when nfs_provisioner_external_server is set.
+        type: str
+        default: lvms-vg-data
+      nfs_provisioner_storage_size:
+        description: >-
+          Size of the in-cluster NFS server backing PVC.
+          Ignored when nfs_provisioner_external_server is set.
+        type: str
+        default: 50Gi
+      nfs_provisioner_name:
+        description: Provisioner name written into the StorageClass.
+        type: str
+        default: nfs-client
+      nfs_provisioner_storage_class_name:
+        description: Name of the RWX StorageClass created by this role.
+        type: str
+        default: nfs-client
+      nfs_provisioner_image:
+        description: Container image for the nfs-subdir-external-provisioner.
+        type: str
+        default: registry.k8s.io/sig-storage/nfs-subdir-external-provisioner:v4.0.2
+      nfs_provisioner_server_image:
+        description: >-
+          Container image for the in-cluster NFS server.
+          Ignored when nfs_provisioner_external_server is set.
+        type: str
+        default: registry.k8s.io/volume-nfs:0.8
+      nfs_provisioner_export_path:
+        description: >-
+          Path exported by the in-cluster NFS server.
+          Ignored when nfs_provisioner_external_server is set.
+        type: str
+        default: /exports
+      nfs_provisioner_wait_timeout:
+        description: Seconds to wait for deployments to become ready.
+        type: int
+        default: 300

roles/nfs_provisioner/meta/main.yml

@@ -0,0 +1,17 @@
+---
+galaxy_info:
+  author: ptoal
+  description: Deploy in-cluster NFS server and provisioner for ReadWriteMany storage on OpenShift
+  license: MIT
+  min_ansible_version: "2.16"
+  platforms:
+    - name: GenericLinux
+      versions:
+        - all
+  galaxy_tags:
+    - openshift
+    - nfs
+    - storage
+    - provisioner
+
+dependencies: []

roles/nfs_provisioner/tasks/main.yml

@@ -0,0 +1,394 @@
+---
+# Deploy nfs-subdir-external-provisioner on OpenShift, backed by either:
+#   (a) an external NFS server (set nfs_provisioner_external_server / nfs_provisioner_external_path)
+#   (b) an in-cluster NFS server pod backed by an LVMS RWO PVC (default)
+#
+# Architecture (in-cluster mode):
+#   - NFS server StatefulSet: backs exports with an LVMS RWO PVC
+#   - Service: exposes NFS server at a stable ClusterIP
+#   - nfs-subdir-external-provisioner: creates PVs on-demand under the export path
+#   - StorageClass: "nfs-client" with ReadWriteMany support
+#
+# Architecture (external mode, nfs_provisioner_external_server != ""):
+#   - In-cluster NFS server is NOT deployed
+#   - nfs-subdir-external-provisioner points directly at the external NFS share
+#   - StorageClass: "nfs-client" with ReadWriteMany support
+#
+# The in-cluster NFS server requires privileged SCC on OpenShift (kernel NFS).
+# All tasks are idempotent (kubernetes.core.k8s state: present).
+
+# ------------------------------------------------------------------
+# Step 1: Namespace and RBAC
+# ------------------------------------------------------------------
+- name: Create NFS provisioner namespace
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: v1
+      kind: Namespace
+      metadata:
+        name: "{{ nfs_provisioner_namespace }}"
+
+- name: Create NFS server ServiceAccount
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: v1
+      kind: ServiceAccount
+      metadata:
+        name: nfs-server
+        namespace: "{{ nfs_provisioner_namespace }}"
+  when: nfs_provisioner_external_server | length == 0
+
+- name: Create NFS provisioner ServiceAccount
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: v1
+      kind: ServiceAccount
+      metadata:
+        name: nfs-provisioner
+        namespace: "{{ nfs_provisioner_namespace }}"
+
+- name: Create ClusterRole to use privileged SCC (NFS server)
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: rbac.authorization.k8s.io/v1
+      kind: ClusterRole
+      metadata:
+        name: nfs-server-scc
+      rules:
+        - apiGroups: [security.openshift.io]
+          resources: [securitycontextconstraints]
+          verbs: [use]
+          resourceNames: [privileged]
+  when: nfs_provisioner_external_server | length == 0
+
+- name: Bind privileged SCC ClusterRole to NFS server ServiceAccount
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: rbac.authorization.k8s.io/v1
+      kind: ClusterRoleBinding
+      metadata:
+        name: nfs-server-scc
+      subjects:
+        - kind: ServiceAccount
+          name: nfs-server
+          namespace: "{{ nfs_provisioner_namespace }}"
+      roleRef:
+        kind: ClusterRole
+        name: nfs-server-scc
+        apiGroup: rbac.authorization.k8s.io
+  when: nfs_provisioner_external_server | length == 0
+
+- name: Create ClusterRole to use hostmount-anyuid SCC (NFS provisioner)
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: rbac.authorization.k8s.io/v1
+      kind: ClusterRole
+      metadata:
+        name: nfs-provisioner-scc
+      rules:
+        - apiGroups: [security.openshift.io]
+          resources: [securitycontextconstraints]
+          verbs: [use]
+          resourceNames: [hostmount-anyuid]
+
+- name: Bind hostmount-anyuid SCC ClusterRole to NFS provisioner ServiceAccount
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: rbac.authorization.k8s.io/v1
+      kind: ClusterRoleBinding
+      metadata:
+        name: nfs-provisioner-scc
+      subjects:
+        - kind: ServiceAccount
+          name: nfs-provisioner
+          namespace: "{{ nfs_provisioner_namespace }}"
+      roleRef:
+        kind: ClusterRole
+        name: nfs-provisioner-scc
+        apiGroup: rbac.authorization.k8s.io
+
+- name: Create ClusterRole for NFS provisioner
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: rbac.authorization.k8s.io/v1
+      kind: ClusterRole
+      metadata:
+        name: nfs-provisioner-runner
+      rules:
+        - apiGroups: [""]
+          resources: [persistentvolumes]
+          verbs: [get, list, watch, create, delete]
+        - apiGroups: [""]
+          resources: [persistentvolumeclaims]
+          verbs: [get, list, watch, update]
+        - apiGroups: [storage.k8s.io]
+          resources: [storageclasses]
+          verbs: [get, list, watch]
+        - apiGroups: [""]
+          resources: [events]
+          verbs: [create, update, patch]
+
+- name: Bind ClusterRole to NFS provisioner ServiceAccount
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: rbac.authorization.k8s.io/v1
+      kind: ClusterRoleBinding
+      metadata:
+        name: run-nfs-provisioner
+      subjects:
+        - kind: ServiceAccount
+          name: nfs-provisioner
+          namespace: "{{ nfs_provisioner_namespace }}"
+      roleRef:
+        kind: ClusterRole
+        name: nfs-provisioner-runner
+        apiGroup: rbac.authorization.k8s.io
+
+- name: Create Role for leader election
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: rbac.authorization.k8s.io/v1
+      kind: Role
+      metadata:
+        name: leader-locking-nfs-provisioner
+        namespace: "{{ nfs_provisioner_namespace }}"
+      rules:
+        - apiGroups: [""]
+          resources: [endpoints]
+          verbs: [get, list, watch, create, update, patch]
+
+- name: Bind leader election Role
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: rbac.authorization.k8s.io/v1
+      kind: RoleBinding
+      metadata:
+        name: leader-locking-nfs-provisioner
+        namespace: "{{ nfs_provisioner_namespace }}"
+      subjects:
+        - kind: ServiceAccount
+          name: nfs-provisioner
+          namespace: "{{ nfs_provisioner_namespace }}"
+      roleRef:
+        kind: Role
+        name: leader-locking-nfs-provisioner
+        apiGroup: rbac.authorization.k8s.io
+
+# ------------------------------------------------------------------
+# Step 2: NFS server backing storage and StatefulSet (in-cluster mode only)
+# ------------------------------------------------------------------
+- name: Create NFS server backing PVC
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: v1
+      kind: PersistentVolumeClaim
+      metadata:
+        name: nfs-server-data
+        namespace: "{{ nfs_provisioner_namespace }}"
+      spec:
+        accessModes:
+          - ReadWriteOnce
+        resources:
+          requests:
+            storage: "{{ nfs_provisioner_storage_size }}"
+        storageClassName: "{{ nfs_provisioner_storage_class }}"
+  when: nfs_provisioner_external_server | length == 0
+
+- name: Deploy NFS server StatefulSet
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: apps/v1
+      kind: StatefulSet
+      metadata:
+        name: nfs-server
+        namespace: "{{ nfs_provisioner_namespace }}"
+      spec:
+        replicas: 1
+        selector:
+          matchLabels:
+            app: nfs-server
+        serviceName: nfs-server
+        template:
+          metadata:
+            labels:
+              app: nfs-server
+          spec:
+            serviceAccountName: nfs-server
+            containers:
+              - name: nfs-server
+                image: "{{ nfs_provisioner_server_image }}"
+                ports:
+                  - name: nfs
+                    containerPort: 2049
+                  - name: mountd
+                    containerPort: 20048
+                  - name: rpcbind
+                    containerPort: 111
+                securityContext:
+                  privileged: true
+                volumeMounts:
+                  - name: nfs-data
+                    mountPath: "{{ nfs_provisioner_export_path }}"
+            volumes:
+              - name: nfs-data
+                persistentVolumeClaim:
+                  claimName: nfs-server-data
+  when: nfs_provisioner_external_server | length == 0
+
+- name: Create NFS server Service
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: v1
+      kind: Service
+      metadata:
+        name: nfs-server
+        namespace: "{{ nfs_provisioner_namespace }}"
+      spec:
+        selector:
+          app: nfs-server
+        ports:
+          - name: nfs
+            port: 2049
+          - name: mountd
+            port: 20048
+          - name: rpcbind
+            port: 111
+  when: nfs_provisioner_external_server | length == 0
+
+# ------------------------------------------------------------------
+# Step 3: Wait for in-cluster NFS server to be ready (in-cluster mode only)
+# ------------------------------------------------------------------
+- name: Wait for NFS server to be ready
+  kubernetes.core.k8s_info:
+    api_version: apps/v1
+    kind: StatefulSet
+    namespace: "{{ nfs_provisioner_namespace }}"
+    name: nfs-server
+  register: __nfs_provisioner_server_status
+  until: >-
+    __nfs_provisioner_server_status.resources | length > 0 and
+    (__nfs_provisioner_server_status.resources[0].status.readyReplicas | default(0)) >= 1
+  retries: "{{ __nfs_provisioner_wait_retries }}"
+  delay: 10
+  when: nfs_provisioner_external_server | length == 0
+
+# ------------------------------------------------------------------
+# Step 4: Resolve NFS server address, then deploy nfs-subdir-external-provisioner
+# ------------------------------------------------------------------
+- name: Set NFS server address (external)
+  ansible.builtin.set_fact:
+    __nfs_provisioner_server_addr: "{{ nfs_provisioner_external_server }}"
+    __nfs_provisioner_server_path: "{{ nfs_provisioner_external_path }}"
+  when: nfs_provisioner_external_server | length > 0
+
+- name: Retrieve in-cluster NFS server ClusterIP
+  kubernetes.core.k8s_info:
+    api_version: v1
+    kind: Service
+    namespace: "{{ nfs_provisioner_namespace }}"
+    name: nfs-server
+  register: __nfs_provisioner_svc
+  when: nfs_provisioner_external_server | length == 0
+
+- name: Set NFS server address (in-cluster)
+  ansible.builtin.set_fact:
+    __nfs_provisioner_server_addr: "{{ __nfs_provisioner_svc.resources[0].spec.clusterIP }}"
+    __nfs_provisioner_server_path: "{{ nfs_provisioner_export_path }}"
+  when: nfs_provisioner_external_server | length == 0
+
+- name: Deploy nfs-subdir-external-provisioner
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: apps/v1
+      kind: Deployment
+      metadata:
+        name: nfs-provisioner
+        namespace: "{{ nfs_provisioner_namespace }}"
+      spec:
+        replicas: 1
+        selector:
+          matchLabels:
+            app: nfs-provisioner
+        strategy:
+          type: Recreate
+        template:
+          metadata:
+            labels:
+              app: nfs-provisioner
+          spec:
+            serviceAccountName: nfs-provisioner
+            containers:
+              - name: nfs-provisioner
+                image: "{{ nfs_provisioner_image }}"
+                env:
+                  - name: PROVISIONER_NAME
+                    value: "{{ nfs_provisioner_name }}"
+                  - name: NFS_SERVER
+                    value: "{{ __nfs_provisioner_server_addr }}"
+                  - name: NFS_PATH
+                    value: "{{ __nfs_provisioner_server_path }}"
+                volumeMounts:
+                  - name: nfs-client-root
+                    mountPath: /persistentvolumes
+            volumes:
+              - name: nfs-client-root
+                nfs:
+                  server: "{{ __nfs_provisioner_server_addr }}"
+                  path: "{{ __nfs_provisioner_server_path }}"
+
+# ------------------------------------------------------------------
+# Step 5: Create StorageClass
+# ------------------------------------------------------------------
+- name: Create NFS StorageClass
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: storage.k8s.io/v1
+      kind: StorageClass
+      metadata:
+        name: "{{ nfs_provisioner_storage_class_name }}"
+      provisioner: "{{ nfs_provisioner_name }}"
+      parameters:
+        archiveOnDelete: "false"
+      reclaimPolicy: Delete
+      volumeBindingMode: Immediate
+
+# ------------------------------------------------------------------
+# Step 6: Wait for provisioner to be ready
+# ------------------------------------------------------------------
+- name: Wait for NFS provisioner deployment to be ready
+  kubernetes.core.k8s_info:
+    api_version: apps/v1
+    kind: Deployment
+    namespace: "{{ nfs_provisioner_namespace }}"
+    name: nfs-provisioner
+  register: __nfs_provisioner_deploy_status
+  until: >-
+    __nfs_provisioner_deploy_status.resources | length > 0 and
+    (__nfs_provisioner_deploy_status.resources[0].status.readyReplicas | default(0)) >= 1
+  retries: "{{ __nfs_provisioner_wait_retries }}"
+  delay: 10
+
+- name: Display NFS provisioner summary
+  ansible.builtin.debug:
+    msg:
+      - "NFS provisioner deployment complete!"
+      - "  Namespace     : {{ nfs_provisioner_namespace }}"
+      - "  NFS server    : {{ __nfs_provisioner_server_addr }}:{{ __nfs_provisioner_server_path }}"
+      - "  Mode          : {{ 'external' if nfs_provisioner_external_server | length > 0 else 'in-cluster (LVMS-backed)' }}"
+      - "  StorageClass  : {{ nfs_provisioner_storage_class_name }} (ReadWriteMany)"

roles/nfs_provisioner/vars/main.yml

@@ -0,0 +1,3 @@
+---
+# Computed internal variables - do not override
+__nfs_provisioner_wait_retries: "{{ (nfs_provisioner_wait_timeout / 10) | int }}"

roles/ocp_service_account/defaults/main.yml

@@ -0,0 +1,6 @@
+---
+# ocp_service_account_name: ""          # required — SA and ClusterRole name
+# ocp_service_account_namespace: ""     # required — namespace for SA and token secret
+# ocp_service_account_cluster_role_rules: []  # required — list of RBAC policy rules
+
+ocp_service_account_create_namespace: true

roles/ocp_service_account/meta/argument_specs.yml

@@ -0,0 +1,29 @@
+---
+argument_specs:
+  main:
+    short_description: Create an OpenShift ServiceAccount with scoped ClusterRole
+    description:
+      - Creates a ServiceAccount, ClusterRole, ClusterRoleBinding, and a
+        long-lived token Secret. The token is registered as
+        __ocp_service_account_token for downstream use.
+    options:
+      ocp_service_account_name:
+        description: Name for the ServiceAccount, ClusterRole, and ClusterRoleBinding.
+        type: str
+        required: true
+      ocp_service_account_namespace:
+        description: Namespace where the ServiceAccount and token Secret are created.
+        type: str
+        required: true
+      ocp_service_account_cluster_role_rules:
+        description: >-
+          List of RBAC policy rules for the ClusterRole.
+          Each item follows the Kubernetes PolicyRule schema
+          (apiGroups, resources, verbs).
+        type: list
+        elements: dict
+        required: true
+      ocp_service_account_create_namespace:
+        description: Whether to create the namespace if it does not exist.
+        type: bool
+        default: true

roles/ocp_service_account/meta/main.yml

@@ -0,0 +1,16 @@
+---
+galaxy_info:
+  author: ptoal
+  description: Create an OpenShift ServiceAccount with ClusterRole and long-lived token
+  license: MIT
+  min_ansible_version: "2.16"
+  platforms:
+    - name: GenericLinux
+      versions:
+        - all
+  galaxy_tags:
+    - openshift
+    - rbac
+    - serviceaccount
+
+dependencies: []

roles/ocp_service_account/tasks/main.yml

@@ -0,0 +1,111 @@
+---
+# Create an OpenShift ServiceAccount with a scoped ClusterRole and long-lived token.
+#
+# Requires: ocp_service_account_name, ocp_service_account_namespace,
+#           ocp_service_account_cluster_role_rules
+#
+# Registers: __ocp_service_account_token (decoded bearer token)
+
+- name: Validate required variables
+  ansible.builtin.assert:
+    that:
+      - ocp_service_account_name | length > 0
+      - ocp_service_account_namespace | length > 0
+      - ocp_service_account_cluster_role_rules | length > 0
+    fail_msg: "ocp_service_account_name, ocp_service_account_namespace, and ocp_service_account_cluster_role_rules are required"
+
+- name: Create namespace {{ ocp_service_account_namespace }}
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: v1
+      kind: Namespace
+      metadata:
+        name: "{{ ocp_service_account_namespace }}"
+  when: ocp_service_account_create_namespace | bool
+
+- name: Create ServiceAccount {{ ocp_service_account_name }}
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: v1
+      kind: ServiceAccount
+      metadata:
+        name: "{{ ocp_service_account_name }}"
+        namespace: "{{ ocp_service_account_namespace }}"
+        labels:
+          app.kubernetes.io/managed-by: ocp-service-account-role
+
+- name: Create ClusterRole {{ ocp_service_account_name }}
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: rbac.authorization.k8s.io/v1
+      kind: ClusterRole
+      metadata:
+        name: "{{ ocp_service_account_name }}"
+        labels:
+          app.kubernetes.io/managed-by: ocp-service-account-role
+      rules: "{{ ocp_service_account_cluster_role_rules }}"
+
+- name: Create ClusterRoleBinding {{ ocp_service_account_name }}
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: rbac.authorization.k8s.io/v1
+      kind: ClusterRoleBinding
+      metadata:
+        name: "{{ ocp_service_account_name }}"
+        labels:
+          app.kubernetes.io/managed-by: ocp-service-account-role
+      roleRef:
+        apiGroup: rbac.authorization.k8s.io
+        kind: ClusterRole
+        name: "{{ ocp_service_account_name }}"
+      subjects:
+        - kind: ServiceAccount
+          name: "{{ ocp_service_account_name }}"
+          namespace: "{{ ocp_service_account_namespace }}"
+
+- name: Create long-lived token Secret for {{ ocp_service_account_name }}
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: v1
+      kind: Secret
+      metadata:
+        name: "{{ ocp_service_account_name }}-token"
+        namespace: "{{ ocp_service_account_namespace }}"
+        labels:
+          app.kubernetes.io/managed-by: ocp-service-account-role
+          app.kubernetes.io/instance: "{{ ocp_service_account_name }}"
+        annotations:
+          kubernetes.io/service-account.name: "{{ ocp_service_account_name }}"
+      type: kubernetes.io/service-account-token
+
+- name: Wait for token to be populated
+  kubernetes.core.k8s_info:
+    api_version: v1
+    kind: Secret
+    namespace: "{{ ocp_service_account_namespace }}"
+    name: "{{ ocp_service_account_name }}-token"
+  register: __ocp_sa_token_secret
+  until: >-
+    __ocp_sa_token_secret.resources | length > 0 and
+    (__ocp_sa_token_secret.resources[0].data.token | default('') | length > 0)
+  retries: 12
+  delay: 5
+
+- name: Register SA token for downstream use
+  ansible.builtin.set_fact:
+    __ocp_service_account_token: "{{ __ocp_sa_token_secret.resources[0].data.token | b64decode }}"
+  no_log: true
+
+- name: Display SA token for vault storage
+  ansible.builtin.debug:
+    msg:
+      - "*** SERVICE ACCOUNT TOKEN — SAVE TO 1PASSWORD ***"
+      - "ServiceAccount: {{ ocp_service_account_name }} ({{ ocp_service_account_namespace }})"
+      - "Vault variable: vault_{{ ocp_service_account_name | regex_replace('-', '_') }}_token"
+      - ""
+      - "Token: {{ __ocp_service_account_token }}"

roles/opnsense_dns_override/README.md

@@ -0,0 +1,61 @@
+# opnsense_dns_override
+
+Manages OPNsense Unbound DNS host overrides (A record) and domain forwards via the `oxlorg.opnsense` collection.
+
+Accepts a list of entries, each specifying either a `host` override or a `forward` rule. All tasks delegate to localhost (OPNsense modules are API-based).
+
+## Requirements
+
+- `oxlorg.opnsense` collection
+- `module_defaults` for `group/oxlorg.opnsense.all` must be set at play level (firewall, api_key, api_secret)
+
+## Role Variables
+
+| Variable | Default | Description |
+|---|---|---|
+| `opnsense_dns_override_entries` | `[]` | List of DNS override entries (see below) |
+
+### Entry format
+
+Each entry in `opnsense_dns_override_entries` requires:
+
+| Field | Required | Description |
+|---|---|---|
+| `type` | yes | `host` for Unbound host override, `forward` for domain forwarding |
+| `value` | yes | Target IP address |
+| `hostname` | host only | Subdomain part (e.g. `api.sno`) |
+| `domain` | yes | Parent domain for host type, or full domain for forward type |
+
+## Example Playbook
+
+```yaml
+- name: Configure OPNsense DNS overrides
+  hosts: gate.toal.ca
+  gather_facts: false
+  connection: local
+
+  module_defaults:
+    group/oxlorg.opnsense.all:
+      firewall: "{{ opnsense_host }}"
+      api_key: "{{ opnsense_api_key }}"
+      api_secret: "{{ opnsense_api_secret }}"
+
+  roles:
+    - role: opnsense_dns_override
+      opnsense_dns_override_entries:
+        - hostname: api.sno
+          domain: openshift.toal.ca
+          value: 192.168.40.10
+          type: host
+        - domain: apps.sno.openshift.toal.ca
+          value: 192.168.40.10
+          type: forward
+```
+
+## License
+
+MIT
+
+## Author
+
+ptoal

roles/opnsense_dns_override/defaults/main.yml

@@ -0,0 +1,26 @@
+---
+# List of DNS override entries to create in OPNsense Unbound.
+#
+# Each entry must have:
+#   type:   "host" for unbound_host (A record override) or
+#           "forward" for unbound_forward (domain forwarding)
+#
+# For type "host":
+#   hostname: subdomain part (e.g. "api.sno")
+#   domain:   parent domain (e.g. "openshift.toal.ca")
+#   value:    target IP address
+#
+# For type "forward":
+#   domain:   full domain to forward (e.g. "apps.sno.openshift.toal.ca")
+#   value:    target IP address
+#
+# Example:
+#   opnsense_dns_override_entries:
+#     - hostname: api.sno
+#       domain: openshift.toal.ca
+#       value: 192.168.40.10
+#       type: host
+#     - domain: apps.sno.openshift.toal.ca
+#       value: 192.168.40.10
+#       type: forward
+opnsense_dns_override_entries: []

roles/opnsense_dns_override/meta/argument_specs.yml

@@ -0,0 +1,17 @@
+---
+argument_specs:
+  main:
+    short_description: Manage OPNsense Unbound DNS overrides
+    description:
+      - Creates Unbound host overrides (A record) and domain forwards
+        in OPNsense via the oxlorg.opnsense collection.
+      - Requires oxlorg.opnsense module_defaults to be set at play level.
+    options:
+      opnsense_dns_override_entries:
+        description: >-
+          List of DNS override entries. Each entry requires C(type) ("host" or "forward"),
+          C(value) (target IP), and either C(hostname)+C(domain) (for host type) or
+          C(domain) (for forward type).
+        type: list
+        elements: dict
+        default: []

roles/opnsense_dns_override/meta/main.yml

@@ -0,0 +1,16 @@
+---
+galaxy_info:
+  author: ptoal
+  description: Manage OPNsense Unbound DNS host overrides and domain forwards
+  license: MIT
+  min_ansible_version: "2.16"
+  platforms:
+    - name: GenericLinux
+      versions:
+        - all
+  galaxy_tags:
+    - opnsense
+    - dns
+    - unbound
+
+dependencies: []

roles/opnsense_dns_override/tasks/main.yml

@@ -0,0 +1,24 @@
+---
+- name: Create Unbound host overrides
+  oxlorg.opnsense.unbound_host:
+    hostname: "{{ item.hostname }}"
+    domain: "{{ item.domain }}"
+    value: "{{ item.value }}"
+    match_fields:
+      - hostname
+      - domain
+    state: present
+  delegate_to: localhost
+  loop: "{{ opnsense_dns_override_entries | selectattr('type', 'eq', 'host') }}"
+  loop_control:
+    label: "{{ item.hostname }}.{{ item.domain }} -> {{ item.value }}"
+
+- name: Create Unbound domain forwards
+  oxlorg.opnsense.unbound_forward:
+    domain: "{{ item.domain }}"
+    target: "{{ item.value }}"
+    state: present
+  delegate_to: localhost
+  loop: "{{ opnsense_dns_override_entries | selectattr('type', 'eq', 'forward') }}"
+  loop_control:
+    label: "{{ item.domain }} -> {{ item.value }}"

roles/proxmox_sno_vm/defaults/main.yml

@@ -1,27 +0,0 @@
----
-# Proxmox connection
-# proxmox_api_host / proxmox_api_port are derived from the 'proxmox_api'
-# inventory host (ansible_host / ansible_port).  Do not set them here.
-proxmox_node: pve1
-proxmox_api_user: ansible@pam
-proxmox_api_token_id: ansible
-proxmox_api_token_secret: "{{ vault_proxmox_token_secret }}"
-proxmox_validate_certs: false
-
-# Storage
-proxmox_storage: local-lvm        # VM disk storage pool
-proxmox_iso_storage: local         # ISO storage pool name (Proxmox)
-proxmox_iso_dir: /var/lib/vz/template/iso  # Filesystem path on proxmox_host
-sno_credentials_dir: "/root/sno-{{ ocp_cluster_name }}"  # Persistent credentials on proxmox_host
-
-# VM specification
-sno_vm_name: "sno-{{ ocp_cluster_name }}"
-sno_cpu: 8
-sno_memory_mb: 32768
-sno_disk_gb: 120
-sno_bridge: vmbr0
-sno_vlan: 40
-sno_mac: ""  # Leave empty for auto-assignment. Set explicitly to pin MAC for static IP.
-
-# VM ID - leave 0 for auto-assign by Proxmox
-sno_vm_id: 0

roles/sno_deploy/defaults/main.yml

@@ -0,0 +1,56 @@
+---
+# --- Proxmox connection ---
+# proxmox_api_host / proxmox_api_port are derived from the 'proxmox_api'
+# inventory host (ansible_host / ansible_port).  Do not set them here.
+proxmox_node: pve1
+proxmox_api_user: ansible@pam
+proxmox_api_token_id: ansible
+proxmox_api_token_secret: "{{ vault_proxmox_token_secret }}"
+proxmox_validate_certs: false
+
+# --- Storage ---
+proxmox_storage: local-lvm
+proxmox_iso_storage: local
+proxmox_iso_dir: /var/lib/vz/template/iso
+sno_credentials_dir: "/root/sno-{{ ocp_cluster_name }}"
+
+# --- VM specification ---
+sno_vm_name: "sno-{{ ocp_cluster_name }}"
+sno_cpu: 8
+sno_memory_mb: 32768
+sno_disk_gb: 120
+sno_pvc_disk_gb: 100
+sno_vnet: ocp
+sno_mac: ""  # populated after VM creation; set here to pin MAC
+sno_vm_id: 0
+
+sno_storage_ip: ""
+sno_storage_ip_prefix_length: 24
+sno_storage_vnet: storage
+sno_storage_mac: ""  # populated after VM creation; set here to pin MAC
+
+# --- Installer ---
+sno_install_dir: "/tmp/sno-{{ ocp_cluster_name }}"
+sno_iso_filename: agent.x86_64.iso
+
+# --- OIDC ---
+oidc_provider_name: keycloak
+oidc_client_id: openshift
+oidc_admin_groups: []
+oidc_ca_cert_file: ""
+
+# --- Keycloak ---
+keycloak_context: ""
+
+# --- cert-manager ---
+sno_deploy_certmanager_channel: "stable-v1"
+sno_deploy_certmanager_source: redhat-operators
+sno_deploy_letsencrypt_email: ""
+sno_deploy_letsencrypt_server: "https://acme-v02.api.letsencrypt.org/directory"
+sno_deploy_letsencrypt_staging_server: "https://acme-staging-v02.api.letsencrypt.org/directory"
+sno_deploy_letsencrypt_use_staging: false
+sno_deploy_certmanager_wait_timeout: 300
+sno_deploy_certificate_wait_timeout: 600
+sno_deploy_certmanager_dns_provider: dnsmadeeasy
+sno_deploy_webhook_image: "ghcr.io/ptoal/cert-manager-webhook-dnsmadeeasy:latest"
+sno_deploy_webhook_group_name: "acme.toal.ca"

roles/sno_deploy/meta/argument_specs.yml

@@ -0,0 +1,127 @@
+---
+argument_specs:
+  main:
+    short_description: Deploy and configure Single Node OpenShift on Proxmox
+    description:
+      - Creates a Proxmox VM, installs SNO via agent-based installer,
+        configures OIDC authentication, deploys cert-manager with LetsEncrypt,
+        and removes the kubeadmin user.
+    options:
+      proxmox_node:
+        description: Proxmox cluster node to create the VM on.
+        type: str
+        default: pve1
+      proxmox_api_user:
+        description: Proxmox API username.
+        type: str
+        default: ansible@pam
+      proxmox_api_token_id:
+        description: Proxmox API token ID.
+        type: str
+        default: ansible
+      proxmox_api_token_secret:
+        description: Proxmox API token secret.
+        type: str
+        required: true
+        no_log: true
+      proxmox_validate_certs:
+        description: Whether to validate TLS certificates for the Proxmox API.
+        type: bool
+        default: false
+      proxmox_storage:
+        description: Proxmox storage pool for VM disks.
+        type: str
+        default: local-lvm
+      proxmox_iso_storage:
+        description: Proxmox storage pool name for ISO images.
+        type: str
+        default: local
+      proxmox_iso_dir:
+        description: Filesystem path on the Proxmox host where ISOs are stored.
+        type: str
+        default: /var/lib/vz/template/iso
+      sno_credentials_dir:
+        description: >-
+          Directory on proxmox_host where kubeconfig and kubeadmin-password
+          are persisted after installation.
+        type: str
+        default: "/root/sno-{{ ocp_cluster_name }}"
+      sno_vm_name:
+        description: Name of the VM in Proxmox.
+        type: str
+        default: "sno-{{ ocp_cluster_name }}"
+      sno_cpu:
+        description: Number of CPU cores for the VM.
+        type: int
+        default: 8
+      sno_memory_mb:
+        description: Memory in megabytes for the VM.
+        type: int
+        default: 32768
+      sno_disk_gb:
+        description: Primary disk size in gigabytes.
+        type: int
+        default: 120
+      sno_vnet:
+        description: Proxmox SDN VNet name for the primary (OCP) NIC.
+        type: str
+        default: ocp
+      sno_mac:
+        description: >-
+          MAC address for the primary NIC. Leave empty for auto-assignment by Proxmox.
+          Set here to pin the MAC across VM recreations.
+        type: str
+        default: ""
+      sno_storage_ip:
+        description: >-
+          IP address for the secondary storage NIC. Leave empty to skip storage
+          interface configuration in agent-config.
+        type: str
+        default: ""
+      sno_storage_ip_prefix_length:
+        description: Prefix length for the storage NIC IP address.
+        type: int
+        default: 24
+      sno_storage_vnet:
+        description: Proxmox SDN VNet name for the secondary storage NIC.
+        type: str
+        default: storage
+      sno_storage_mac:
+        description: >-
+          MAC address for the storage NIC. Leave empty for auto-assignment by Proxmox.
+          Set here to pin the MAC across VM recreations.
+        type: str
+        default: ""
+      sno_vm_id:
+        description: Proxmox VM ID. Set to 0 for auto-assignment.
+        type: int
+        default: 0
+      sno_install_dir:
+        description: Local directory for openshift-install working files.
+        type: str
+        default: "/tmp/sno-{{ ocp_cluster_name }}"
+      sno_iso_filename:
+        description: Filename for the agent-based installer ISO.
+        type: str
+        default: agent.x86_64.iso
+      oidc_provider_name:
+        description: Identity provider name shown on OpenShift login page.
+        type: str
+        default: keycloak
+      oidc_client_id:
+        description: OIDC client ID registered in Keycloak.
+        type: str
+        default: openshift
+      oidc_admin_groups:
+        description: List of OIDC groups to grant cluster-admin via ClusterRoleBinding.
+        type: list
+        elements: str
+        default: []
+      sno_deploy_letsencrypt_email:
+        description: Email address for LetsEncrypt ACME account registration.
+        type: str
+        required: true
+      sno_deploy_certmanager_channel:
+        description: OLM subscription channel for cert-manager operator.
+        type: str
+        default: "stable-v1"

roles/sno_deploy/meta/main.yml

@@ -0,0 +1,19 @@
+---
+galaxy_info:
+  author: ptoal
+  description: Deploy and configure Single Node OpenShift (SNO) on Proxmox
+  license: MIT
+  min_ansible_version: "2.16"
+  platforms:
+    - name: GenericLinux
+      versions:
+        - all
+  galaxy_tags:
+    - proxmox
+    - openshift
+    - sno
+    - vm
+    - oidc
+    - certmanager
+
+dependencies: []

roles/sno_deploy/tasks/configure_certmanager.yml

@@ -0,0 +1,542 @@
+---
+# Install cert-manager operator and configure LetsEncrypt certificates.
+#
+# Installs the Red Hat cert-manager operator via OLM, creates a ClusterIssuer
+# for LetsEncrypt with DNS-01 challenges via DNS Made Easy, and provisions
+# certificates for the ingress wildcard and API server.
+
+# ------------------------------------------------------------------
+# Step 1: Install cert-manager operator via OLM
+# ------------------------------------------------------------------
+- name: Ensure cert-manager-operator namespace exists
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: v1
+      kind: Namespace
+      metadata:
+        name: cert-manager-operator
+
+- name: Create OperatorGroup for cert-manager-operator
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: operators.coreos.com/v1
+      kind: OperatorGroup
+      metadata:
+        name: cert-manager-operator
+        namespace: cert-manager-operator
+      spec:
+        targetNamespaces:
+          - cert-manager-operator
+
+- name: Subscribe to cert-manager operator
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: operators.coreos.com/v1alpha1
+      kind: Subscription
+      metadata:
+        name: openshift-cert-manager-operator
+        namespace: cert-manager-operator
+      spec:
+        channel: "{{ sno_deploy_certmanager_channel }}"
+        installPlanApproval: Automatic
+        name: openshift-cert-manager-operator
+        source: "{{ sno_deploy_certmanager_source }}"
+        sourceNamespace: openshift-marketplace
+
+# ------------------------------------------------------------------
+# Step 2: Wait for cert-manager to be ready
+# ------------------------------------------------------------------
+- name: Wait for cert-manager CRDs to be available
+  kubernetes.core.k8s_info:
+    api_version: apiextensions.k8s.io/v1
+    kind: CustomResourceDefinition
+    name: certificates.cert-manager.io
+  register: __sno_deploy_certmanager_crd
+  until: __sno_deploy_certmanager_crd.resources | length > 0
+  retries: "{{ (sno_deploy_certmanager_wait_timeout / 10) | int }}"
+  delay: 10
+
+- name: Wait for cert-manager deployment to be ready
+  kubernetes.core.k8s_info:
+    api_version: apps/v1
+    kind: Deployment
+    namespace: cert-manager
+    name: cert-manager
+  register: __sno_deploy_certmanager_deploy
+  until: >-
+    __sno_deploy_certmanager_deploy.resources | length > 0 and
+    (__sno_deploy_certmanager_deploy.resources[0].status.readyReplicas | default(0)) >= 1
+  retries: "{{ (sno_deploy_certmanager_wait_timeout / 10) | int }}"
+  delay: 10
+
+# ------------------------------------------------------------------
+# Step 3: Create DNS Made Easy API credentials for DNS-01 challenges
+# ------------------------------------------------------------------
+- name: Create DNS Made Easy API credentials secret
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: v1
+      kind: Secret
+      metadata:
+        name: dme-api-credentials
+        namespace: cert-manager
+      type: Opaque
+      stringData:
+        api-key: "{{ dme_account_key }}"
+        secret-key: "{{ dme_account_secret }}"
+  no_log: true
+
+# ------------------------------------------------------------------
+# Step 4: Deploy DNS Made Easy webhook solver
+# ------------------------------------------------------------------
+- name: Create webhook namespace
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: v1
+      kind: Namespace
+      metadata:
+        name: cert-manager-webhook-dnsmadeeasy
+
+- name: Create webhook ServiceAccount
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: v1
+      kind: ServiceAccount
+      metadata:
+        name: cert-manager-webhook-dnsmadeeasy
+        namespace: cert-manager-webhook-dnsmadeeasy
+
+- name: Create webhook ClusterRole
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: rbac.authorization.k8s.io/v1
+      kind: ClusterRole
+      metadata:
+        name: cert-manager-webhook-dnsmadeeasy
+      rules:
+        - apiGroups: [""]
+          resources: ["secrets"]
+          verbs: ["get", "list", "watch"]
+        - apiGroups: ["flowcontrol.apiserver.k8s.io"]
+          resources: ["flowschemas", "prioritylevelconfigurations"]
+          verbs: ["list", "watch"]
+
+- name: Create webhook ClusterRoleBinding
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: rbac.authorization.k8s.io/v1
+      kind: ClusterRoleBinding
+      metadata:
+        name: cert-manager-webhook-dnsmadeeasy
+      roleRef:
+        apiGroup: rbac.authorization.k8s.io
+        kind: ClusterRole
+        name: cert-manager-webhook-dnsmadeeasy
+      subjects:
+        - kind: ServiceAccount
+          name: cert-manager-webhook-dnsmadeeasy
+          namespace: cert-manager-webhook-dnsmadeeasy
+
+- name: Create auth-delegator ClusterRoleBinding for webhook
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: rbac.authorization.k8s.io/v1
+      kind: ClusterRoleBinding
+      metadata:
+        name: cert-manager-webhook-dnsmadeeasy:auth-delegator
+      roleRef:
+        apiGroup: rbac.authorization.k8s.io
+        kind: ClusterRole
+        name: system:auth-delegator
+      subjects:
+        - kind: ServiceAccount
+          name: cert-manager-webhook-dnsmadeeasy
+          namespace: cert-manager-webhook-dnsmadeeasy
+
+- name: Create authentication-reader RoleBinding for webhook
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: rbac.authorization.k8s.io/v1
+      kind: RoleBinding
+      metadata:
+        name: cert-manager-webhook-dnsmadeeasy:webhook-authentication-reader
+        namespace: kube-system
+      roleRef:
+        apiGroup: rbac.authorization.k8s.io
+        kind: Role
+        name: extension-apiserver-authentication-reader
+      subjects:
+        - kind: ServiceAccount
+          name: cert-manager-webhook-dnsmadeeasy
+          namespace: cert-manager-webhook-dnsmadeeasy
+
+- name: Create domain-solver ClusterRole for cert-manager
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: rbac.authorization.k8s.io/v1
+      kind: ClusterRole
+      metadata:
+        name: cert-manager-webhook-dnsmadeeasy:domain-solver
+      rules:
+        - apiGroups: ["{{ sno_deploy_webhook_group_name }}"]
+          resources: ["*"]
+          verbs: ["create"]
+
+- name: Bind domain-solver to cert-manager ServiceAccount
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: rbac.authorization.k8s.io/v1
+      kind: ClusterRoleBinding
+      metadata:
+        name: cert-manager-webhook-dnsmadeeasy:domain-solver
+      roleRef:
+        apiGroup: rbac.authorization.k8s.io
+        kind: ClusterRole
+        name: cert-manager-webhook-dnsmadeeasy:domain-solver
+      subjects:
+        - kind: ServiceAccount
+          name: cert-manager
+          namespace: cert-manager
+
+- name: Create self-signed Issuer for webhook TLS
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: cert-manager.io/v1
+      kind: Issuer
+      metadata:
+        name: cert-manager-webhook-dnsmadeeasy-selfsign
+        namespace: cert-manager-webhook-dnsmadeeasy
+      spec:
+        selfSigned: {}
+
+- name: Create webhook TLS certificate
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: cert-manager.io/v1
+      kind: Certificate
+      metadata:
+        name: cert-manager-webhook-dnsmadeeasy-tls
+        namespace: cert-manager-webhook-dnsmadeeasy
+      spec:
+        secretName: cert-manager-webhook-dnsmadeeasy-tls
+        duration: 8760h
+        renewBefore: 720h
+        issuerRef:
+          name: cert-manager-webhook-dnsmadeeasy-selfsign
+          kind: Issuer
+        dnsNames:
+          - cert-manager-webhook-dnsmadeeasy
+          - cert-manager-webhook-dnsmadeeasy.cert-manager-webhook-dnsmadeeasy
+          - cert-manager-webhook-dnsmadeeasy.cert-manager-webhook-dnsmadeeasy.svc
+
+- name: Deploy webhook solver
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: apps/v1
+      kind: Deployment
+      metadata:
+        name: cert-manager-webhook-dnsmadeeasy
+        namespace: cert-manager-webhook-dnsmadeeasy
+      spec:
+        replicas: 1
+        selector:
+          matchLabels:
+            app: cert-manager-webhook-dnsmadeeasy
+        template:
+          metadata:
+            labels:
+              app: cert-manager-webhook-dnsmadeeasy
+          spec:
+            serviceAccountName: cert-manager-webhook-dnsmadeeasy
+            containers:
+              - name: webhook
+                image: "{{ sno_deploy_webhook_image }}"
+                args:
+                  - --tls-cert-file=/tls/tls.crt
+                  - --tls-private-key-file=/tls/tls.key
+                  - --secure-port=8443
+                ports:
+                  - containerPort: 8443
+                    name: https
+                    protocol: TCP
+                env:
+                  - name: GROUP_NAME
+                    value: "{{ sno_deploy_webhook_group_name }}"
+                livenessProbe:
+                  httpGet:
+                    path: /healthz
+                    port: https
+                    scheme: HTTPS
+                  initialDelaySeconds: 5
+                  periodSeconds: 10
+                readinessProbe:
+                  httpGet:
+                    path: /healthz
+                    port: https
+                    scheme: HTTPS
+                  initialDelaySeconds: 5
+                  periodSeconds: 10
+                resources:
+                  requests:
+                    cpu: 10m
+                    memory: 32Mi
+                  limits:
+                    memory: 64Mi
+                volumeMounts:
+                  - name: certs
+                    mountPath: /tls
+                    readOnly: true
+            volumes:
+              - name: certs
+                secret:
+                  secretName: cert-manager-webhook-dnsmadeeasy-tls
+
+- name: Create webhook Service
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: v1
+      kind: Service
+      metadata:
+        name: cert-manager-webhook-dnsmadeeasy
+        namespace: cert-manager-webhook-dnsmadeeasy
+      spec:
+        type: ClusterIP
+        ports:
+          - port: 443
+            targetPort: https
+            protocol: TCP
+            name: https
+        selector:
+          app: cert-manager-webhook-dnsmadeeasy
+
+- name: Register webhook APIService
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: apiregistration.k8s.io/v1
+      kind: APIService
+      metadata:
+        name: "v1alpha1.{{ sno_deploy_webhook_group_name }}"
+      spec:
+        group: "{{ sno_deploy_webhook_group_name }}"
+        groupPriorityMinimum: 1000
+        versionPriority: 15
+        service:
+          name: cert-manager-webhook-dnsmadeeasy
+          namespace: cert-manager-webhook-dnsmadeeasy
+        version: v1alpha1
+        insecureSkipTLSVerify: true
+
+- name: Wait for webhook deployment to be ready
+  kubernetes.core.k8s_info:
+    api_version: apps/v1
+    kind: Deployment
+    namespace: cert-manager-webhook-dnsmadeeasy
+    name: cert-manager-webhook-dnsmadeeasy
+  register: __sno_deploy_webhook_deploy
+  until: >-
+    __sno_deploy_webhook_deploy.resources | length > 0 and
+    (__sno_deploy_webhook_deploy.resources[0].status.readyReplicas | default(0)) >= 1
+  retries: 30
+  delay: 10
+
+# ------------------------------------------------------------------
+# Step 5: Create ClusterIssuer for LetsEncrypt
+# ------------------------------------------------------------------
+- name: Create LetsEncrypt ClusterIssuer
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: cert-manager.io/v1
+      kind: ClusterIssuer
+      metadata:
+        name: letsencrypt-production
+      spec:
+        acme:
+          email: "{{ sno_deploy_letsencrypt_email }}"
+          server: "{{ __sno_deploy_letsencrypt_server_url }}"
+          privateKeySecretRef:
+            name: letsencrypt-production-account-key
+          solvers:
+            - dns01:
+                webhook:
+                  groupName: "{{ sno_deploy_webhook_group_name }}"
+                  solverName: dnsmadeeasy
+                  config:
+                    apiKeySecretRef:
+                      name: dme-api-credentials
+                      key: api-key
+                    secretKeySecretRef:
+                      name: dme-api-credentials
+                      key: secret-key
+
+- name: Wait for ClusterIssuer to be ready
+  kubernetes.core.k8s_info:
+    api_version: cert-manager.io/v1
+    kind: ClusterIssuer
+    name: letsencrypt-production
+  register: __sno_deploy_clusterissuer
+  until: >-
+    __sno_deploy_clusterissuer.resources | length > 0 and
+    (__sno_deploy_clusterissuer.resources[0].status.conditions | default([])
+     | selectattr('type', '==', 'Ready')
+     | selectattr('status', '==', 'True') | list | length > 0)
+  retries: 12
+  delay: 10
+
+# ------------------------------------------------------------------
+# Step 6: Create Certificate resources
+# ------------------------------------------------------------------
+- name: Create apps wildcard certificate
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: cert-manager.io/v1
+      kind: Certificate
+      metadata:
+        name: apps-wildcard-cert
+        namespace: openshift-ingress
+      spec:
+        secretName: apps-wildcard-tls
+        issuerRef:
+          name: letsencrypt-production
+          kind: ClusterIssuer
+        dnsNames:
+          - "{{ __sno_deploy_apps_wildcard }}"
+        duration: 2160h
+        renewBefore: 720h
+
+- name: Create API server certificate
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: cert-manager.io/v1
+      kind: Certificate
+      metadata:
+        name: api-server-cert
+        namespace: openshift-config
+      spec:
+        secretName: api-server-tls
+        issuerRef:
+          name: letsencrypt-production
+          kind: ClusterIssuer
+        dnsNames:
+          - "{{ __sno_deploy_api_hostname }}"
+        duration: 2160h
+        renewBefore: 720h
+
+# ------------------------------------------------------------------
+# Step 7: Wait for certificates to be issued
+# ------------------------------------------------------------------
+- name: Wait for apps wildcard certificate to be ready
+  kubernetes.core.k8s_info:
+    api_version: cert-manager.io/v1
+    kind: Certificate
+    namespace: openshift-ingress
+    name: apps-wildcard-cert
+  register: __sno_deploy_apps_cert
+  until: >-
+    __sno_deploy_apps_cert.resources | length > 0 and
+    (__sno_deploy_apps_cert.resources[0].status.conditions | default([])
+     | selectattr('type', '==', 'Ready')
+     | selectattr('status', '==', 'True') | list | length > 0)
+  retries: "{{ (sno_deploy_certificate_wait_timeout / 10) | int }}"
+  delay: 10
+
+- name: Wait for API server certificate to be ready
+  kubernetes.core.k8s_info:
+    api_version: cert-manager.io/v1
+    kind: Certificate
+    namespace: openshift-config
+    name: api-server-cert
+  register: __sno_deploy_api_cert
+  until: >-
+    __sno_deploy_api_cert.resources | length > 0 and
+    (__sno_deploy_api_cert.resources[0].status.conditions | default([])
+     | selectattr('type', '==', 'Ready')
+     | selectattr('status', '==', 'True') | list | length > 0)
+  retries: "{{ (sno_deploy_certificate_wait_timeout / 10) | int }}"
+  delay: 10
+
+# ------------------------------------------------------------------
+# Step 8: Patch IngressController and APIServer to use the certs
+# ------------------------------------------------------------------
+- name: Patch default IngressController to use LetsEncrypt cert
+  kubernetes.core.k8s:
+    state: present
+    merge_type: merge
+    definition:
+      apiVersion: operator.openshift.io/v1
+      kind: IngressController
+      metadata:
+        name: default
+        namespace: openshift-ingress-operator
+      spec:
+        defaultCertificate:
+          name: apps-wildcard-tls
+
+- name: Patch APIServer to use LetsEncrypt cert
+  kubernetes.core.k8s:
+    state: present
+    merge_type: merge
+    definition:
+      apiVersion: config.openshift.io/v1
+      kind: APIServer
+      metadata:
+        name: cluster
+      spec:
+        servingCerts:
+          namedCertificates:
+            - names:
+                - "{{ __sno_deploy_api_hostname }}"
+              servingCertificate:
+                name: api-server-tls
+
+# ------------------------------------------------------------------
+# Step 9: Wait for rollouts
+# ------------------------------------------------------------------
+- name: Wait for API server to begin restart
+  ansible.builtin.pause:
+    seconds: 30
+
+- name: Wait for router pods to restart with new cert
+  kubernetes.core.k8s_info:
+    api_version: apps/v1
+    kind: Deployment
+    namespace: openshift-ingress
+    name: router-default
+  register: __sno_deploy_router
+  until: >-
+    __sno_deploy_router.resources is defined and
+    __sno_deploy_router.resources | length > 0 and
+    (__sno_deploy_router.resources[0].status.updatedReplicas | default(0)) ==
+    (__sno_deploy_router.resources[0].status.replicas | default(1)) and
+    (__sno_deploy_router.resources[0].status.readyReplicas | default(0)) ==
+    (__sno_deploy_router.resources[0].status.replicas | default(1))
+  retries: 60
+  delay: 10
+
+- name: Display cert-manager configuration summary
+  ansible.builtin.debug:
+    msg:
+      - "cert-manager configuration complete!"
+      - "  ClusterIssuer : letsencrypt-production"
+      - "  Apps wildcard : {{ __sno_deploy_apps_wildcard }}"
+      - "  API cert      : {{ __sno_deploy_api_hostname }}"
+    verbosity: 1

roles/sno_deploy/tasks/configure_oidc.yml

@@ -0,0 +1,145 @@
+---
+# Configure OpenShift OAuth with Keycloak OIDC.
+#
+# Prerequisites:
+#   - SNO cluster installed and accessible
+#   - Keycloak OIDC client created (Play 5 in deploy_openshift.yml)
+#   - KUBECONFIG environment variable set or oc_kubeconfig defined
+
+# ------------------------------------------------------------------
+# Secret: Keycloak client secret in openshift-config namespace
+# ------------------------------------------------------------------
+- name: Set OIDC client secret value
+  ansible.builtin.set_fact:
+    __sno_deploy_oidc_client_secret_value: >-
+      {{ hostvars[inventory_hostname]['__oidc_client_secret']
+         | default(vault_oidc_client_secret) }}
+  no_log: true
+
+- name: Create Keycloak client secret in openshift-config
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: v1
+      kind: Secret
+      metadata:
+        name: "{{ __sno_deploy_oidc_secret_name }}"
+        namespace: openshift-config
+      type: Opaque
+      stringData:
+        clientSecret: "{{ __sno_deploy_oidc_client_secret_value }}"
+  no_log: false
+
+# ------------------------------------------------------------------
+# CA bundle: only needed when Keycloak uses a private/internal CA
+# ------------------------------------------------------------------
+- name: Create CA bundle ConfigMap for Keycloak TLS
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: v1
+      kind: ConfigMap
+      metadata:
+        name: "{{ __sno_deploy_oidc_ca_configmap_name }}"
+        namespace: openshift-config
+      data:
+        ca.crt: "{{ lookup('ansible.builtin.file', oidc_ca_cert_file) }}"
+  when: oidc_ca_cert_file | default('') | length > 0
+
+# ------------------------------------------------------------------
+# OAuth cluster resource: add/replace Keycloak IdP entry
+# ------------------------------------------------------------------
+- name: Get current OAuth cluster configuration
+  kubernetes.core.k8s_info:
+    api_version: config.openshift.io/v1
+    kind: OAuth
+    name: cluster
+  register: __sno_deploy_current_oauth
+
+- name: Build Keycloak OIDC identity provider definition
+  ansible.builtin.set_fact:
+    __sno_deploy_new_idp: >-
+      {{
+        {
+          'name': oidc_provider_name,
+          'mappingMethod': 'claim',
+          'type': 'OpenID',
+          'openID': (
+            {
+              'clientID': oidc_client_id,
+              'clientSecret': {'name': __sno_deploy_oidc_secret_name},
+              'issuer': __sno_deploy_oidc_issuer,
+              'claims': {
+                'preferredUsername': ['preferred_username'],
+                'name': ['name'],
+                'email': ['email'],
+                'groups': ['groups']
+              }
+            } | combine(
+              (oidc_ca_cert_file | default('') | length > 0) | ternary(
+                {'ca': {'name': __sno_deploy_oidc_ca_configmap_name}}, {}
+              )
+            )
+          )
+        }
+      }}
+
+- name: Build updated identity providers list
+  ansible.builtin.set_fact:
+    __sno_deploy_updated_idps: >-
+      {{
+        (__sno_deploy_current_oauth.resources[0].spec.identityProviders | default([])
+         | selectattr('name', '!=', oidc_provider_name) | list)
+        + [__sno_deploy_new_idp]
+      }}
+
+- name: Apply updated OAuth cluster configuration
+  kubernetes.core.k8s:
+    state: present
+    merge_type: merge
+    definition:
+      apiVersion: config.openshift.io/v1
+      kind: OAuth
+      metadata:
+        name: cluster
+      spec:
+        identityProviders: "{{ __sno_deploy_updated_idps }}"
+
+- name: Wait for OAuth deployment to roll out
+  ansible.builtin.command:
+    cmd: "{{ __sno_deploy_oc }} rollout status deployment/oauth-openshift -n openshift-authentication --timeout=300s --insecure-skip-tls-verify"
+  changed_when: false
+
+# ------------------------------------------------------------------
+# ClusterRoleBinding: grant cluster-admin to OIDC admin groups
+# ------------------------------------------------------------------
+- name: Create ClusterRoleBinding for OIDC admin groups
+  kubernetes.core.k8s:
+    state: present
+    definition:
+      apiVersion: rbac.authorization.k8s.io/v1
+      kind: ClusterRoleBinding
+      metadata:
+        name: "oidc-{{ item | regex_replace('[^a-zA-Z0-9-]', '-') }}-cluster-admin"
+      roleRef:
+        apiGroup: rbac.authorization.k8s.io
+        kind: ClusterRole
+        name: cluster-admin
+      subjects:
+        - apiGroup: rbac.authorization.k8s.io
+          kind: Group
+          name: "{{ item }}"
+  loop: "{{ oidc_admin_groups }}"
+  when: oidc_admin_groups | length > 0
+
+- name: Display post-configuration summary
+  ansible.builtin.debug:
+    msg:
+      - "OpenShift OIDC configuration complete!"
+      - "  Provider : {{ oidc_provider_name }}"
+      - "  Issuer   : {{ __sno_deploy_oidc_issuer }}"
+      - "  Console  : https://console-openshift-console.apps.{{ ocp_cluster_name }}.{{ ocp_base_domain }}"
+      - "  Login    : https://oauth-openshift.apps.{{ ocp_cluster_name }}.{{ ocp_base_domain }}"
+      - ""
+      - "Note: OAuth pods are restarting — login may be unavailable for ~2 minutes."
+    verbosity: 1

roles/sno_deploy/tasks/create_vm.yml

@@ -1,16 +1,21 @@
 ---
-# Create a Proxmox VM for Single Node OpenShift on VLAN40 (192.168.40.0/24).
+# Create a Proxmox VM for Single Node OpenShift.
 # Uses q35 machine type with UEFI (required for SNO / RHCOS).
-# An empty ide2 CD-ROM slot is created here so the boot order can reference it;
-# the deploy_openshift.yml play loads the actual ISO into it after generation.
+# An empty ide2 CD-ROM slot is created for the agent installer ISO.
 
 - name: Build net0 string
   ansible.builtin.set_fact:
-    # Proxmox net format: model[=macaddr],bridge=<bridge>[,tag=<vlan>]
-    _sno_net0: >-
+    __sno_deploy_net0: >-
       virtio{{
         '=' + sno_mac if sno_mac | length > 0 else ''
-      }},bridge={{ sno_bridge }},tag={{ sno_vlan }}
+      }},bridge={{ sno_vnet }}
+
+- name: Build net1 (storage) string
+  ansible.builtin.set_fact:
+    __sno_deploy_net1: >-
+      virtio{{
+        '=' + sno_storage_mac if sno_storage_mac | length > 0 else ''
+      }},bridge={{ sno_storage_vnet }}
 
 - name: Create SNO VM in Proxmox
   community.proxmox.proxmox_kvm:
@@ -36,15 +41,17 @@
       pre_enrolled_keys: false
     scsi:
       scsi0: "{{ proxmox_storage }}:{{ sno_disk_gb }},format=raw,iothread=1,cache=writeback"
+      scsi1: "{{ proxmox_storage }}:{{ sno_pvc_disk_gb }},format=raw,iothread=1,cache=writeback"
     scsihw: virtio-scsi-single
     ide:
       ide2: none,media=cdrom
     net:
-      net0: "{{ _sno_net0 }}"
+      net0: "{{ __sno_deploy_net0 }}"
+      net1: "{{ __sno_deploy_net1 }}"
     boot: "order=scsi0;ide2"
     onboot: true
     state: present
-  register: proxmox_vm_result
+  register: __sno_deploy_vm_result
 
 - name: Retrieve VM info
   community.proxmox.proxmox_vm_info:
@@ -58,27 +65,37 @@
     name: "{{ sno_vm_name }}"
     type: qemu
     config: current
-  register: proxmox_vm_info
+  register: __sno_deploy_vm_info
   retries: 5
 
 - name: Set VM ID fact for subsequent plays
   ansible.builtin.set_fact:
-    sno_vm_id: "{{ proxmox_vm_info.proxmox_vms[0].vmid }}"
+    sno_vm_id: "{{ __sno_deploy_vm_info.proxmox_vms[0].vmid }}"
     cacheable: true
 
 - name: Extract MAC address from VM config
   ansible.builtin.set_fact:
-    # net0 format: virtio=52:54:00:xx:xx:xx,bridge=vmbr0,tag=40
     sno_mac: >-
-      {{ proxmox_vm_info.proxmox_vms[0].config.net0
+      {{ __sno_deploy_vm_info.proxmox_vms[0].config.net0
          | regex_search('([0-9A-Fa-f]{2}(?::[0-9A-Fa-f]{2}){5})', '\1')
          | first }}
     cacheable: true
   when: sno_mac | length == 0
 
+- name: Extract storage MAC address from VM config
+  ansible.builtin.set_fact:
+    sno_storage_mac: >-
+      {{ __sno_deploy_vm_info.proxmox_vms[0].config.net1
+         | regex_search('([0-9A-Fa-f]{2}(?::[0-9A-Fa-f]{2}){5})', '\1')
+         | first }}
+    cacheable: true
+  when: sno_storage_mac | length == 0
+
 - name: Display VM details
   ansible.builtin.debug:
     msg:
-      - "VM Name : {{ sno_vm_name }}"
-      - "VM ID   : {{ sno_vm_id }}"
-      - "MAC     : {{ sno_mac }}"
+      - "VM Name     : {{ sno_vm_name }}"
+      - "VM ID       : {{ sno_vm_id }}"
+      - "MAC (net0)  : {{ sno_mac }}"
+      - "MAC (net1)  : {{ sno_storage_mac }}"
+    verbosity: 1

roles/sno_deploy/tasks/delete_kubeadmin.yml

@@ -0,0 +1,52 @@
+---
+# Delete the kubeadmin user after OIDC is configured and admin groups
+# have cluster-admin. This is a security best practice.
+#
+# Safety checks:
+#   1. Verify at least one group in oidc_admin_groups is configured
+#   2. Verify ClusterRoleBindings exist for those groups
+#   3. Verify the OAuth deployment is ready (OIDC login is available)
+#   4. Only then delete the kubeadmin secret
+
+- name: Fail if no admin groups are configured
+  ansible.builtin.fail:
+    msg: >-
+      Cannot delete kubeadmin: oidc_admin_groups is empty.
+      At least one OIDC group must have cluster-admin before kubeadmin can be removed.
+  when: oidc_admin_groups | length == 0
+
+- name: Verify OIDC admin ClusterRoleBindings exist
+  kubernetes.core.k8s_info:
+    api_version: rbac.authorization.k8s.io/v1
+    kind: ClusterRoleBinding
+    name: "oidc-{{ item | regex_replace('[^a-zA-Z0-9-]', '-') }}-cluster-admin"
+  loop: "{{ oidc_admin_groups }}"
+  register: __sno_deploy_admin_crbs
+  failed_when: __sno_deploy_admin_crbs.resources | length == 0
+
+- name: Verify OAuth deployment is ready
+  kubernetes.core.k8s_info:
+    api_version: apps/v1
+    kind: Deployment
+    namespace: openshift-authentication
+    name: oauth-openshift
+  register: __sno_deploy_oauth_status
+  failed_when: >-
+    __sno_deploy_oauth_status.resources | length == 0 or
+    (__sno_deploy_oauth_status.resources[0].status.readyReplicas | default(0)) < 1
+
+- name: Delete kubeadmin secret
+  kubernetes.core.k8s:
+    api_version: v1
+    kind: Secret
+    namespace: kube-system
+    name: kubeadmin
+    state: absent
+  register: __sno_deploy_kubeadmin_deleted
+
+- name: Display kubeadmin deletion result
+  ansible.builtin.debug:
+    msg: >-
+      {{ 'kubeadmin user deleted successfully. Login is now only available via OIDC.'
+         if __sno_deploy_kubeadmin_deleted.changed
+         else 'kubeadmin was already deleted.' }}

roles/sno_deploy/tasks/install.yml

@@ -0,0 +1,389 @@
+---
+# Generate Agent ISO and deploy SNO (agent-based installer).
+#
+# Uses `openshift-install agent create image` — no SaaS API, no SSO required.
+# The pull secret is the only Red Hat credential needed.
+# Credentials (kubeconfig, kubeadmin-password) are generated locally under
+# sno_install_dir/auth/ by openshift-install itself.
+#
+# Idempotency: If the cluster API is already responding, all install steps
+# are skipped.  Credentials on Proxmox host are never overwritten once saved.
+
+# ------------------------------------------------------------------
+# Step 0: Ensure sno_vm_id and sno_mac are populated.
+# These are set as cacheable facts by create_vm.yml, but in ephemeral
+# EEs or when running --tags sno_deploy_install alone the cache is empty.
+# ------------------------------------------------------------------
+- name: Retrieve VM info from Proxmox (needed when fact cache is empty)
+  community.proxmox.proxmox_vm_info:
+    api_host: "{{ hostvars['proxmox_api']['ansible_host'] }}"
+    api_user: "{{ proxmox_api_user }}"
+    api_port: "{{ hostvars['proxmox_api']['ansible_port'] }}"
+    api_token_id: "{{ proxmox_api_token_id }}"
+    api_token_secret: "{{ proxmox_api_token_secret }}"
+    validate_certs: "{{ proxmox_validate_certs }}"
+    node: "{{ proxmox_node }}"
+    name: "{{ sno_vm_name }}"
+    type: qemu
+    config: current
+  register: __sno_deploy_vm_info
+  when: (sno_vm_id | default('')) == '' or (sno_mac | default('')) == ''
+
+- name: Set sno_vm_id and sno_mac from live Proxmox query
+  ansible.builtin.set_fact:
+    sno_vm_id: "{{ __sno_deploy_vm_info.proxmox_vms[0].vmid }}"
+    sno_mac: >-
+      {{ __sno_deploy_vm_info.proxmox_vms[0].config.net0
+         | regex_search('([0-9A-Fa-f]{2}(?::[0-9A-Fa-f]{2}){5})', '\1')
+         | first }}
+    cacheable: true
+  when: __sno_deploy_vm_info is not skipped
+
+# ------------------------------------------------------------------
+# Step 0b: Check if OpenShift is already deployed and responding.
+# If the API is reachable, skip ISO generation, boot, and install.
+# ------------------------------------------------------------------
+- name: Check if OpenShift cluster is already responding
+  ansible.builtin.uri:
+    url: "https://api.{{ ocp_cluster_name }}.{{ ocp_base_domain }}:6443/readyz"
+    method: GET
+    validate_certs: false
+    status_code: [200, 401, 403]
+    timeout: 10
+  register: __sno_deploy_cluster_alive
+  ignore_errors: true
+
+- name: Set cluster deployed flag
+  ansible.builtin.set_fact:
+    __sno_deploy_cluster_deployed: "{{ __sno_deploy_cluster_alive is success }}"
+
+- name: Display cluster status
+  ansible.builtin.debug:
+    msg: >-
+      {{ 'OpenShift cluster is already deployed and responding — skipping install steps.'
+         if __sno_deploy_cluster_deployed | bool
+         else 'OpenShift cluster is not yet deployed — proceeding with installation.' }}
+
+- name: Ensure local install directories exist
+  ansible.builtin.file:
+    path: "{{ item }}"
+    state: directory
+    mode: "0750"
+  loop:
+    - "{{ sno_install_dir }}"
+    - "{{ sno_install_dir }}/auth"
+
+# ------------------------------------------------------------------
+# Step 0c: When cluster is already deployed, ensure a valid kubeconfig
+# exists so post-install tasks can authenticate to the API.
+# Try in order: local file → Proxmox host backup → SSH to SNO node.
+# After obtaining a kubeconfig, validate it against the API and fall
+# through to the next source if credentials are expired.
+# ------------------------------------------------------------------
+- name: Check if local kubeconfig already exists
+  ansible.builtin.stat:
+    path: "{{ __sno_deploy_kubeconfig }}"
+  register: __sno_deploy_local_kubeconfig
+  when: __sno_deploy_cluster_deployed | bool
+
+- name: Validate local kubeconfig against API
+  ansible.builtin.command:
+    cmd: "oc whoami --kubeconfig={{ __sno_deploy_kubeconfig }} --insecure-skip-tls-verify"
+  register: __sno_deploy_local_kubeconfig_valid
+  ignore_errors: true
+  changed_when: false
+  when:
+    - __sno_deploy_cluster_deployed | bool
+    - __sno_deploy_local_kubeconfig.stat.exists | default(false)
+
+- name: Check if kubeconfig exists on Proxmox host
+  ansible.builtin.stat:
+    path: "{{ sno_credentials_dir }}/kubeconfig"
+  delegate_to: proxmox_host
+  register: __sno_deploy_proxmox_kubeconfig
+  when:
+    - __sno_deploy_cluster_deployed | bool
+    - not (__sno_deploy_local_kubeconfig.stat.exists | default(false)) or
+      (__sno_deploy_local_kubeconfig_valid is failed)
+
+- name: Recover kubeconfig from Proxmox host
+  ansible.builtin.fetch:
+    src: "{{ sno_credentials_dir }}/kubeconfig"
+    dest: "{{ __sno_deploy_kubeconfig }}"
+    flat: true
+  delegate_to: proxmox_host
+  when:
+    - __sno_deploy_cluster_deployed | bool
+    - not (__sno_deploy_local_kubeconfig.stat.exists | default(false)) or
+      (__sno_deploy_local_kubeconfig_valid is failed)
+    - __sno_deploy_proxmox_kubeconfig.stat.exists | default(false)
+
+- name: Validate recovered Proxmox kubeconfig against API
+  ansible.builtin.command:
+    cmd: "oc whoami --kubeconfig={{ __sno_deploy_kubeconfig }} --insecure-skip-tls-verify"
+  register: __sno_deploy_proxmox_kubeconfig_valid
+  ignore_errors: true
+  changed_when: false
+  when:
+    - __sno_deploy_cluster_deployed | bool
+    - not (__sno_deploy_local_kubeconfig.stat.exists | default(false)) or
+      (__sno_deploy_local_kubeconfig_valid is failed)
+    - __sno_deploy_proxmox_kubeconfig.stat.exists | default(false)
+
+- name: Set flag - need SSH recovery
+  ansible.builtin.set_fact:
+    __sno_deploy_need_ssh_recovery: >-
+      {{
+        (__sno_deploy_cluster_deployed | bool) and
+        (
+          (not (__sno_deploy_local_kubeconfig.stat.exists | default(false)) and
+           not (__sno_deploy_proxmox_kubeconfig.stat.exists | default(false)))
+          or
+          ((__sno_deploy_local_kubeconfig_valid | default({})) is failed and
+           (__sno_deploy_proxmox_kubeconfig_valid | default({})) is failed)
+          or
+          (not (__sno_deploy_local_kubeconfig.stat.exists | default(false)) and
+           (__sno_deploy_proxmox_kubeconfig_valid | default({})) is failed)
+        )
+      }}
+
+- name: Recover kubeconfig from SNO node via SSH
+  ansible.builtin.command:
+    cmd: >-
+      ssh -o StrictHostKeyChecking=no core@{{ sno_ip }}
+      sudo cat /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/node-kubeconfigs/lb-ext.kubeconfig
+  register: __sno_deploy_recovered_kubeconfig
+  when: __sno_deploy_need_ssh_recovery | bool
+
+- name: Write recovered kubeconfig from SNO node
+  ansible.builtin.copy:
+    content: "{{ __sno_deploy_recovered_kubeconfig.stdout }}"
+    dest: "{{ __sno_deploy_kubeconfig }}"
+    mode: "0600"
+  when:
+    - __sno_deploy_recovered_kubeconfig is not skipped
+    - __sno_deploy_recovered_kubeconfig.rc == 0
+
+- name: Update kubeconfig backup on Proxmox host
+  ansible.builtin.copy:
+    src: "{{ __sno_deploy_kubeconfig }}"
+    dest: "{{ sno_credentials_dir }}/kubeconfig"
+    mode: "0600"
+    backup: true
+  delegate_to: proxmox_host
+  when:
+    - __sno_deploy_recovered_kubeconfig is not skipped
+    - __sno_deploy_recovered_kubeconfig.rc == 0
+
+- name: Fail if no valid kubeconfig could be obtained
+  ansible.builtin.fail:
+    msg: >-
+      Cluster is deployed but no valid kubeconfig could be obtained.
+      Tried: local file, Proxmox host ({{ sno_credentials_dir }}/kubeconfig),
+      and SSH to core@{{ sno_ip }}. Cannot proceed with post-install tasks.
+  when:
+    - __sno_deploy_need_ssh_recovery | bool
+    - __sno_deploy_recovered_kubeconfig is skipped or __sno_deploy_recovered_kubeconfig.rc != 0
+
+# ------------------------------------------------------------------
+# Step 1: Check whether a fresh ISO already exists on Proxmox
+#         AND the local openshift-install state dir is intact.
+# ------------------------------------------------------------------
+- name: Check if ISO already exists on Proxmox and is less than 24 hours old
+  ansible.builtin.stat:
+    path: "{{ proxmox_iso_dir }}/{{ sno_iso_filename }}"
+    get_checksum: false
+  delegate_to: proxmox_host
+  register: __sno_deploy_iso_stat
+  when: not __sno_deploy_cluster_deployed | bool
+
+- name: Check if local openshift-install state directory exists
+  ansible.builtin.stat:
+    path: "{{ sno_install_dir }}/.openshift_install_state"
+    get_checksum: false
+  register: __sno_deploy_state_stat
+  when: not __sno_deploy_cluster_deployed | bool
+
+- name: Set fact - skip ISO build if recent ISO exists on Proxmox and local state is intact
+  ansible.builtin.set_fact:
+    __sno_deploy_iso_fresh: >-
+      {{
+        not (__sno_deploy_cluster_deployed | bool) and
+        __sno_deploy_iso_stat.stat.exists | default(false) and
+        (now(utc=true).timestamp() | int - __sno_deploy_iso_stat.stat.mtime | default(0) | int) < 86400 and
+        __sno_deploy_state_stat.stat.exists | default(false)
+      }}
+
+# ------------------------------------------------------------------
+# Step 2: Get openshift-install binary
+# Always ensure the binary is present — needed for both ISO generation
+# and wait-for-install-complete regardless of __sno_deploy_iso_fresh.
+# ------------------------------------------------------------------
+- name: Download openshift-install tarball
+  ansible.builtin.get_url:
+    url: "https://mirror.openshift.com/pub/openshift-v4/clients/ocp/stable-{{ ocp_version }}/openshift-install-linux.tar.gz"
+    dest: "{{ sno_install_dir }}/openshift-install-{{ ocp_version }}.tar.gz"
+    mode: "0644"
+    checksum: "{{ ocp_install_checksum | default(omit) }}"
+  register: __sno_deploy_install_tarball
+  when: not __sno_deploy_cluster_deployed | bool
+
+- name: Extract openshift-install binary
+  ansible.builtin.unarchive:
+    src: "{{ sno_install_dir }}/openshift-install-{{ ocp_version }}.tar.gz"
+    dest: "{{ sno_install_dir }}"
+    remote_src: false
+    include:
+      - openshift-install
+  when: not __sno_deploy_cluster_deployed | bool and (__sno_deploy_install_tarball.changed or not (sno_install_dir ~ '/openshift-install') is file)
+
+- name: Download openshift-client tarball
+  ansible.builtin.get_url:
+    url: "https://mirror.openshift.com/pub/openshift-v4/clients/ocp/stable-{{ ocp_version }}/openshift-client-linux.tar.gz"
+    dest: "{{ sno_install_dir }}/openshift-client-{{ ocp_version }}.tar.gz"
+    mode: "0644"
+    checksum: "{{ ocp_client_checksum | default(omit) }}"
+  register: __sno_deploy_client_tarball
+  when: not __sno_deploy_cluster_deployed | bool
+
+- name: Extract oc binary
+  ansible.builtin.unarchive:
+    src: "{{ sno_install_dir }}/openshift-client-{{ ocp_version }}.tar.gz"
+    dest: "{{ sno_install_dir }}"
+    remote_src: false
+    include:
+      - oc
+  when: not __sno_deploy_cluster_deployed | bool and (__sno_deploy_client_tarball.changed or not (sno_install_dir ~ '/oc') is file)
+
+# ------------------------------------------------------------------
+# Step 3: Template agent installer config files (skipped if ISO is fresh)
+# ------------------------------------------------------------------
+- name: Template install-config.yaml
+  ansible.builtin.template:
+    src: install-config.yaml.j2
+    dest: "{{ sno_install_dir }}/install-config.yaml"
+    mode: "0640"
+  when: not __sno_deploy_cluster_deployed | bool and not __sno_deploy_iso_fresh | bool
+  no_log: true
+
+- name: Template agent-config.yaml
+  ansible.builtin.template:
+    src: agent-config.yaml.j2
+    dest: "{{ sno_install_dir }}/agent-config.yaml"
+    mode: "0640"
+  when: not __sno_deploy_cluster_deployed | bool and not __sno_deploy_iso_fresh | bool
+
+# ------------------------------------------------------------------
+# Step 4: Generate discovery ISO (skipped if ISO is fresh)
+# ------------------------------------------------------------------
+- name: Generate agent-based installer ISO
+  ansible.builtin.command:
+    cmd: "{{ sno_install_dir }}/openshift-install agent create image --dir {{ sno_install_dir }}"
+  when: not __sno_deploy_cluster_deployed | bool and not __sno_deploy_iso_fresh | bool
+
+# ------------------------------------------------------------------
+# Step 5: Upload ISO to Proxmox and attach to VM
+# ------------------------------------------------------------------
+- name: Copy discovery ISO to Proxmox ISO storage
+  ansible.builtin.copy:
+    src: "{{ sno_install_dir }}/{{ sno_iso_filename }}"
+    dest: "{{ proxmox_iso_dir }}/{{ sno_iso_filename }}"
+    mode: "0644"
+  delegate_to: proxmox_host
+  when: not __sno_deploy_cluster_deployed | bool and not __sno_deploy_iso_fresh | bool
+
+- name: Attach ISO to VM as CDROM
+  ansible.builtin.command:
+    cmd: "qm set {{ sno_vm_id }} --ide2 {{ proxmox_iso_storage }}:iso/{{ sno_iso_filename }},media=cdrom"
+  delegate_to: proxmox_host
+  changed_when: true
+  when: not __sno_deploy_cluster_deployed | bool
+
+- name: Ensure boot order prefers disk, falls back to CDROM
+  ansible.builtin.command:
+    cmd: "qm set {{ sno_vm_id }} --boot order=scsi0;ide2"
+  delegate_to: proxmox_host
+  changed_when: true
+  when: not __sno_deploy_cluster_deployed | bool
+
+# ------------------------------------------------------------------
+# Step 6: Boot the VM
+# ------------------------------------------------------------------
+- name: Start SNO VM
+  community.proxmox.proxmox_kvm:
+    api_host: "{{ hostvars['proxmox_api']['ansible_host'] }}"
+    api_user: "{{ proxmox_api_user }}"
+    api_port: "{{ hostvars['proxmox_api']['ansible_port'] }}"
+    api_token_id: "{{ proxmox_api_token_id }}"
+    api_token_secret: "{{ proxmox_api_token_secret }}"
+    validate_certs: "{{ proxmox_validate_certs }}"
+    node: "{{ proxmox_node }}"
+    name: "{{ sno_vm_name }}"
+    state: started
+  when: not __sno_deploy_cluster_deployed | bool
+
+# ------------------------------------------------------------------
+# Step 7: Wait for installation to complete (~60-90 min)
+# ------------------------------------------------------------------
+- name: Wait for SNO installation to complete
+  ansible.builtin.command:
+    cmd: "{{ sno_install_dir }}/openshift-install agent wait-for install-complete --dir {{ sno_install_dir }} --log-level=info"
+  async: 5400
+  poll: 30
+  when: not __sno_deploy_cluster_deployed | bool
+
+# ------------------------------------------------------------------
+# Step 8: Persist credentials to Proxmox host
+# Only copy if credentials do not already exist on the remote host,
+# to prevent overwriting valid credentials on re-runs.
+# ------------------------------------------------------------------
+- name: Create credentials directory on Proxmox host
+  ansible.builtin.file:
+    path: "{{ sno_credentials_dir }}"
+    state: directory
+    mode: "0700"
+  delegate_to: proxmox_host
+
+- name: Check if credentials already exist on Proxmox host
+  ansible.builtin.stat:
+    path: "{{ sno_credentials_dir }}/kubeadmin-password"
+  delegate_to: proxmox_host
+  register: __sno_deploy_remote_creds
+
+- name: Copy kubeconfig to Proxmox host
+  ansible.builtin.copy:
+    src: "{{ sno_install_dir }}/auth/kubeconfig"
+    dest: "{{ sno_credentials_dir }}/kubeconfig"
+    mode: "0600"
+    backup: true
+  delegate_to: proxmox_host
+  when: not __sno_deploy_remote_creds.stat.exists
+
+- name: Copy kubeadmin-password to Proxmox host
+  ansible.builtin.copy:
+    src: "{{ sno_install_dir }}/auth/kubeadmin-password"
+    dest: "{{ sno_credentials_dir }}/kubeadmin-password"
+    mode: "0600"
+    backup: true
+  delegate_to: proxmox_host
+  when: not __sno_deploy_remote_creds.stat.exists
+
+# ------------------------------------------------------------------
+# Step 9: Eject CDROM so the VM never boots the agent ISO again
+# ------------------------------------------------------------------
+- name: Eject CDROM after successful installation
+  ansible.builtin.command:
+    cmd: "qm set {{ sno_vm_id }} --ide2 none,media=cdrom"
+  delegate_to: proxmox_host
+  changed_when: true
+  when: not __sno_deploy_cluster_deployed | bool
+
+- name: Display post-install info
+  ansible.builtin.debug:
+    msg:
+      - "SNO installation complete!"
+      - "API URL        : https://api.{{ ocp_cluster_name }}.{{ ocp_base_domain }}:6443"
+      - "Console        : https://console-openshift-console.apps.{{ ocp_cluster_name }}.{{ ocp_base_domain }}"
+      - "Kubeconfig     : {{ sno_credentials_dir }}/kubeconfig  (on proxmox_host)"
+      - "kubeadmin pass : {{ sno_credentials_dir }}/kubeadmin-password  (on proxmox_host)"
+    verbosity: 1

roles/sno_deploy/tasks/main.yml

@@ -0,0 +1,41 @@
+---
+# Entry point for the sno_deploy role.
+#
+# Each phase is gated by tags so individual steps can be run with --tags.
+# When invoked from deploy_openshift.yml, individual task files are
+# called directly via include_role + tasks_from to control play ordering.
+
+- name: Create SNO VM in Proxmox
+  ansible.builtin.include_tasks:
+    file: create_vm.yml
+    apply:
+      tags: sno_deploy_vm
+  tags: sno_deploy_vm
+
+- name: Install SNO via agent-based installer
+  ansible.builtin.include_tasks:
+    file: install.yml
+    apply:
+      tags: sno_deploy_install
+  tags: sno_deploy_install
+
+- name: Configure OpenShift OAuth with OIDC
+  ansible.builtin.include_tasks:
+    file: configure_oidc.yml
+    apply:
+      tags: sno_deploy_oidc
+  tags: sno_deploy_oidc
+
+- name: Configure cert-manager and LetsEncrypt certificates
+  ansible.builtin.include_tasks:
+    file: configure_certmanager.yml
+    apply:
+      tags: sno_deploy_certmanager
+  tags: sno_deploy_certmanager
+
+- name: Delete kubeadmin user
+  ansible.builtin.include_tasks:
+    file: delete_kubeadmin.yml
+    apply:
+      tags: sno_deploy_delete_kubeadmin
+  tags: sno_deploy_delete_kubeadmin

roles/sno_deploy/templates/agent-config.yaml.j2

@@ -1,6 +1,6 @@
 ---
 # Generated by Ansible — do not edit by hand
-# Source: playbooks/templates/agent-config.yaml.j2
+# Source: roles/sno_deploy/templates/agent-config.yaml.j2
 apiVersion: v1alpha1
 kind: AgentConfig
 metadata:
@@ -11,6 +11,10 @@ hosts:
     interfaces:
       - name: primary
         macAddress: "{{ sno_mac }}"
+{% if sno_storage_ip | length > 0 %}
+      - name: storage
+        macAddress: "{{ sno_storage_mac }}"
+{% endif %}
     networkConfig:
       interfaces:
         - name: primary
@@ -23,6 +27,18 @@ hosts:
               - ip: {{ sno_ip }}
                 prefix-length: {{ sno_prefix_length }}
             dhcp: false
+{% if sno_storage_ip | length > 0 %}
+        - name: storage
+          type: ethernet
+          state: up
+          mac-address: "{{ sno_storage_mac }}"
+          ipv4:
+            enabled: true
+            address:
+              - ip: {{ sno_storage_ip }}
+                prefix-length: {{ sno_storage_ip_prefix_length }}
+            dhcp: false
+{% endif %}
       dns-resolver:
         config:
           server:

roles/sno_deploy/templates/install-config.yaml.j2

@@ -1,6 +1,6 @@
 ---
 # Generated by Ansible — do not edit by hand
-# Source: playbooks/templates/install-config.yaml.j2
+# Source: roles/sno_deploy/templates/install-config.yaml.j2
 apiVersion: v1
 baseDomain: {{ ocp_base_domain }}
 metadata:
@@ -23,5 +23,5 @@ controlPlane:
 platform:
   none: {}
 pullSecret: |
-  {{ ocp_pull_secret | ansible.builtin.to_json }}
+  {{ vault_ocp_pull_secret | ansible.builtin.to_json }}
 sshKey: "{{ ocp_ssh_public_key }}"

roles/sno_deploy/vars/main.yml

@@ -0,0 +1,13 @@
+---
+# Computed internal variables - do not override
+__sno_deploy_oc: "{{ oc_binary | default('oc') }}"
+__sno_deploy_kubeconfig: "{{ sno_install_dir }}/auth/kubeconfig"
+__sno_deploy_oidc_secret_name: "{{ oidc_provider_name | lower }}"
+__sno_deploy_oidc_ca_configmap_name: "{{ oidc_provider_name }}-oidc-ca-bundle"
+__sno_deploy_oidc_redirect_uri: "https://oauth-openshift.apps.{{ ocp_cluster_name }}.{{ ocp_base_domain }}/oauth2callback/{{ oidc_provider_name }}"
+__sno_deploy_oidc_issuer: "{{ keycloak_url }}{{ keycloak_context }}/realms/{{ keycloak_realm }}"
+__sno_deploy_api_hostname: "api.{{ ocp_cluster_name }}.{{ ocp_base_domain }}"
+__sno_deploy_apps_wildcard: "*.apps.{{ ocp_cluster_name }}.{{ ocp_base_domain }}"
+__sno_deploy_letsencrypt_server_url: >-
+  {{ sno_deploy_letsencrypt_use_staging | bool |
+     ternary(sno_deploy_letsencrypt_staging_server, sno_deploy_letsencrypt_server) }}

roles/toal-common/README.md

@@ -1,38 +0,0 @@
-Role Name
-=========
-
-A brief description of the role goes here.
-
-Requirements
-------------
-
-Any pre-requisites that may not be covered by Ansible itself or the role should be mentioned here. For instance, if the role uses the EC2 module, it may be a good idea to mention in this section that the boto package is required.
-
-Role Variables
---------------
-
-A description of the settable variables for this role should go here, including any variables that are in defaults/main.yml, vars/main.yml, and any variables that can/should be set via parameters to the role. Any variables that are read from other roles and/or the global scope (ie. hostvars, group vars, etc.) should be mentioned here as well.
-
-Dependencies
-------------
-
-A list of other roles hosted on Galaxy should go here, plus any details in regards to parameters that may need to be set for other roles, or variables that are used from other roles.
-
-Example Playbook
-----------------
-
-Including an example of how to use your role (for instance, with variables passed in as parameters) is always nice for users too:
-
-    - hosts: servers
-      roles:
-         - { role: username.rolename, x: 42 }
-
-License
--------
-
-BSD
-
-Author Information
-------------------
-
-An optional section for the role authors to include contact information, or a website (HTML is not allowed).

roles/toal-common/defaults/main.yml

@@ -1,2 +0,0 @@
----
-# defaults file for toal-common

roles/toal-common/files/hello.txt

@@ -1 +0,0 @@
-Hello World

roles/toal-common/handlers/main.yml

@@ -1,14 +0,0 @@
----
-# handlers file for toal-common
-
-  - name: Ovirt Agent Restart
-    service:
-      name: ovirt-guest-agent
-      state: restarted
-    when: ansible_virtualization_type == "RHEV"
-
-  - name: Qemu Agent Restart
-    service:
-      name: qemu-guest-agent
-      state: restarted
-    when: ansible_virtualization_type == "RHEV"

roles/toal-common/meta/main.yml

@@ -1,57 +0,0 @@
-galaxy_info:
-  author: your name
-  description: your description
-  company: your company (optional)
-
-  # If the issue tracker for your role is not on github, uncomment the
-  # next line and provide a value
-  # issue_tracker_url: http://example.com/issue/tracker
-
-  # Some suggested licenses:
-  # - BSD (default)
-  # - MIT
-  # - GPLv2
-  # - GPLv3
-  # - Apache
-  # - CC-BY
-  license: license (GPLv2, CC-BY, etc)
-
-  min_ansible_version: 1.2
-
-  # If this a Container Enabled role, provide the minimum Ansible Container version.
-  # min_ansible_container_version:
-
-  # Optionally specify the branch Galaxy will use when accessing the GitHub
-  # repo for this role. During role install, if no tags are available,
-  # Galaxy will use this branch. During import Galaxy will access files on
-  # this branch. If Travis integration is configured, only notifications for this
-  # branch will be accepted. Otherwise, in all cases, the repo's default branch
-  # (usually master) will be used.
-  #github_branch:
-
-  #
-  # platforms is a list of platforms, and each platform has a name and a list of versions.
-  #
-  # platforms:
-  # - name: Fedora
-  #   versions:
-  #   - all
-  #   - 25
-  # - name: SomePlatform
-  #   versions:
-  #   - all
-  #   - 1.0
-  #   - 7
-  #   - 99.99
-
-  galaxy_tags: []
-    # List tags for your role here, one per line. A tag is a keyword that describes
-    # and categorizes the role. Users find roles by searching for tags. Be sure to
-    # remove the '[]' above, if you add tags to this list.
-    #
-    # NOTE: A tag is limited to a single word comprised of alphanumeric characters.
-    #       Maximum 20 tags per role.
-
-dependencies: []
-  # List your role dependencies here, one per line. Be sure to remove the '[]' above,
-  # if you add dependencies to this list.

roles/toal-common/tasks/main.yml

@@ -1,49 +0,0 @@
----
-# Ensure that virtual guests have the guest tools installed.
-# TODO: Refactor to make cleaner, and more DRY
-- block:
-  - name: Guest Tools Repository
-    rhsm_repository:
-      name: rhel-7-server-rh-common-rpms
-      state: present
-    when:
-     - ansible_distribution_major_version == '7'
-
-  - name: Install ovirt-guest-agent on RHV Guests
-    yum:
-      name: ovirt-guest-agent
-      state: present
-    notify: Ovirt Agent Restart
-    when:
-     - ansible_distribution_major_version == '7'
-
-  - name: Guest Tools Repository
-    rhsm_repository:
-      name: rhel-8-for-x86_64-appstream-rpms
-      state: present
-    when:
-     - ansible_distribution_major_version == '8'
-
-  - name: Install qemu-guest agent on RHEL8 Guest
-    yum:
-      name: qemu-guest-agent
-      state: present
-    notify: Qemu Agent Restart
-    when:
-     - ansible_distribution_major_version == '8'
-
-  when:
-    - ansible_os_family == "RedHat"
-    - ansible_virtualization_type == "RHEV"
-
-- name: Install katello-agent on Satellite managed systems
-  yum:
-    name: katello-agent
-    state: present
-  when: foreman is defined
-
-- name: Install insights-client on RHEL systems
-  yum:
-    name: insights-client
-    state: present
-  when: ansible_distribution == "RedHat"

roles/toal-common/tests/inventory

@@ -1,2 +0,0 @@
-localhost
-

roles/toal-common/tests/test.yml

@@ -1,5 +0,0 @@
----
-- hosts: localhost
-  remote_user: root
-  roles:
-    - toal-common

roles/toal-common/vars/main.yml

@@ -1,2 +0,0 @@
----
-# vars file for toal-common

templates/claude-templates.md

@@ -0,0 +1,490 @@
+# Claude Templates — On-Demand Reference
+
+> **Do NOT read this file at session start.** Read it only when you need to write a summary, handoff, decision record, or subagent output. This file is referenced from CLAUDE.md.
+
+---
+
+## Template 1: Source Document Summary
+
+**Use when:** Processing any input document (client brief, research report, requirements doc, existing proposal)
+
+**Write to:** `./docs/summaries/source-[filename].md`
+
+```markdown
+# Source Summary: [Original Document Name]
+**Processed:** [YYYY-MM-DD]
+**Source Path:** [exact file path]
+**Archived From:** [original path, if moved to docs/archive/]
+**Document Type:** [brief / requirements / research / proposal / transcript / other]
+**Confidence:** [high = I understood everything / medium = some interpretation needed / low = significant gaps]
+
+## Exact Numbers & Metrics
+<!-- List EVERY specific number, dollar amount, percentage, date, count, measurement.
+     Do NOT round. Do NOT paraphrase. Copy exactly as stated in source. -->
+- [metric]: [exact value] (page/section reference if available)
+- [metric]: [exact value]
+
+## Key Facts (Confirmed)
+<!-- Only include facts explicitly stated in the document. Tag source. -->
+- [fact] — stated in [section/page]
+- [fact] — stated in [section/page]
+
+## Requirements & Constraints
+<!-- Use IF/THEN/BUT/EXCEPT format to preserve conditional logic -->
+- REQUIREMENT: [what is needed]
+  - CONDITION: [when/if this applies]
+  - CONSTRAINT: [limitation or exception]
+  - PRIORITY: [must-have / should-have / nice-to-have / stated by whom]
+
+## Decisions Referenced
+<!-- Any decisions mentioned in the document -->
+- DECISION: [what was decided]
+  - RATIONALE: [why, as stated in document]
+  - ALTERNATIVES MENTIONED: [what else was considered]
+  - DECIDED BY: [who, if stated]
+
+## Relationships to Other Documents
+<!-- How this document connects to other known project documents -->
+- SUPPORTS: [other document/decision it reinforces]
+- CONTRADICTS: [other document/decision it conflicts with]
+- DEPENDS ON: [other document/decision it requires]
+- UPDATES: [other document/decision it supersedes]
+
+## Open Questions & Ambiguities
+<!-- Things that are NOT resolved in this document -->
+- UNCLEAR: [what is ambiguous] — needs clarification from [whom]
+- ASSUMED: [interpretation made] — verify with [whom]
+- MISSING: [information referenced but not provided]
+
+## Verbatim Quotes Worth Preserving
+<!-- 2-5 direct quotes that capture stakeholder language, priorities, or constraints
+     These are critical for proposals — use the client's own words back to them -->
+- "[exact quote]" — [speaker/author], [context]
+```
+
+---
+
+## Template 2: Analysis / Research Summary
+
+**Use when:** Completing competitive analysis, market research, technical evaluation
+
+**Write to:** `./docs/summaries/analysis-[topic].md`
+
+```markdown
+# Analysis Summary: [Topic]
+**Completed:** [YYYY-MM-DD]
+**Analysis Type:** [competitive / market / technical / financial / feasibility]
+**Sources Used:** [list source paths or URLs]
+**Confidence:** [high / medium / low — and WHY this confidence level]
+
+## Core Finding (One Sentence)
+[Single sentence: the most important conclusion]
+
+## Evidence Base
+<!-- Specific data points supporting the finding. Exact numbers only. -->
+| Data Point | Value | Source | Date of Data |
+|-----------|-------|--------|-------------|
+| [metric]  | [exact value] | [source] | [date] |
+
+## Detailed Findings
+### Finding 1: [Name]
+- WHAT: [the finding]
+- SO WHAT: [why it matters for this project]
+- EVIDENCE: [specific supporting data]
+- CONFIDENCE: [high/medium/low]
+
+### Finding 2: [Name]
+[same structure]
+
+## Conditional Conclusions
+<!-- Use IF/THEN format -->
+- IF [condition], THEN [conclusion], BECAUSE [evidence]
+- IF [alternative condition], THEN [different conclusion]
+
+## What This Analysis Does NOT Cover
+<!-- Explicit scope boundaries to prevent future sessions from over-interpreting -->
+- [topic not addressed and why]
+- [data not available]
+
+## Recommended Next Steps
+1. [action] — priority [high/medium/low], depends on [what]
+2. [action]
+```
+
+---
+
+## Template 3: Decision Record
+
+**Use when:** Any significant decision is made during a session
+
+**Write to:** `./docs/summaries/decision-[number]-[topic].md`
+
+```markdown
+# Decision Record: [Short Title]
+**Decision ID:** DR-[sequential number]
+**Date:** [YYYY-MM-DD]
+**Status:** CONFIRMED / PROVISIONAL / REQUIRES_VALIDATION
+
+## Decision
+[One clear sentence: what was decided]
+
+## Context
+[2-3 sentences: what situation prompted this decision]
+
+## Rationale
+- CHOSE [option] BECAUSE: [specific reasons with data]
+- REJECTED [alternative 1] BECAUSE: [specific reasons]
+- REJECTED [alternative 2] BECAUSE: [specific reasons]
+
+## Quantified Impact
+- [metric affected]: [expected change with numbers]
+- [cost/time/resource implication]: [specific figures]
+
+## Conditions & Constraints
+- VALID IF: [conditions under which this decision holds]
+- REVISIT IF: [triggers that should cause reconsideration]
+- DEPENDS ON: [upstream decisions or facts this relies on]
+
+## Stakeholder Input
+- [name/role]: [their stated position, if known]
+
+## Downstream Effects
+- AFFECTS: [what other decisions, documents, or deliverables this impacts]
+- REQUIRES UPDATE TO: [specific files or deliverables that need revision]
+```
+
+---
+
+## Template 4: Session Handoff
+
+**Use when:** A session is ending (context limit approaching OR phase complete)
+
+**Write to:** `./docs/summaries/handoff-[YYYY-MM-DD]-[topic].md`
+
+**LIFECYCLE**: After writing a new handoff, move the PREVIOUS handoff to `docs/archive/handoffs/`.
+
+```markdown
+# Session Handoff: [Topic]
+**Date:** [YYYY-MM-DD]
+**Session Duration:** [approximate]
+**Session Focus:** [one sentence]
+**Context Usage at Handoff:** [estimated percentage if known]
+
+## What Was Accomplished
+<!-- Be specific. Include file paths for every output. -->
+1. [task completed] → output at `[file path]`
+2. [task completed] → output at `[file path]`
+
+## Exact State of Work in Progress
+<!-- If anything is mid-stream, describe exactly where it stopped -->
+- [work item]: completed through [specific point], next step is [specific action]
+- [work item]: blocked on [specific issue]
+
+## Decisions Made This Session
+<!-- Reference decision records if created, otherwise summarize here -->
+- DR-[number]: [decision] (see `./docs/summaries/decision-[file]`)
+- [Ad-hoc decision]: [what] BECAUSE [why] — STATUS: [confirmed/provisional]
+
+## Key Numbers Generated or Discovered This Session
+<!-- Every metric, estimate, or figure produced. Exact values. -->
+- [metric]: [value] — [context for where/how this was derived]
+
+## Conditional Logic Established
+<!-- Any IF/THEN/BUT/EXCEPT reasoning that future sessions must respect -->
+- IF [condition] THEN [approach] BECAUSE [rationale]
+
+## Files Created or Modified
+| File Path | Action | Description |
+|-----------|--------|-------------|
+| `[path]`  | Created | [what it contains] |
+| `[path]`  | Modified | [what changed and why] |
+
+## What the NEXT Session Should Do
+<!-- Ordered, specific instructions. The next session starts by reading this. -->
+1. **First**: [specific action with file paths]
+2. **Then**: [specific action]
+3. **Then**: [specific action]
+
+## Open Questions Requiring User Input
+<!-- Do NOT proceed on these without explicit user confirmation -->
+- [ ] [question] — impacts [what downstream deliverable]
+- [ ] [question]
+
+## Assumptions That Need Validation
+<!-- Things treated as true this session but not confirmed -->
+- ASSUMED: [assumption] — validate by [method/person]
+
+## What NOT to Re-Read
+<!-- Prevent the next session from wasting context on already-processed material -->
+- `[file path]` — already summarized in `[summary file path]`
+
+## Files to Load Next Session
+<!-- Explicit index of what the next session should read. Acts as progressive disclosure index layer. -->
+- `[file path]` — needed for [reason]
+- `[file path]` — needed for [reason]
+```
+
+---
+
+## Template 5: Project Brief (Initial Setup)
+
+**Use when:** Creating the 00-project-brief.md at project start
+
+**Write to:** `./docs/summaries/00-project-brief.md`
+
+```markdown
+# Project Brief: [Project Name]
+**Created:** [YYYY-MM-DD]
+**Last Updated:** [YYYY-MM-DD]
+
+## Client
+- **Name:** [client name]
+- **Industry:** [industry]
+- **Size:** [employee count / revenue if known]
+- **Relationship:** [through AutomatonsX / Lagrange Data / direct / other]
+- **Key Contacts:** [names and roles if known]
+
+## Engagement
+- **Type:** [proposal / workshop / competitive analysis / agent development / hybrid]
+- **Scope:** [one paragraph description]
+- **Target Deliverable:** [specific output expected]
+- **Timeline:** [deadline if known]
+- **Budget Context:** [if known — exact figures]
+
+## Input Documents
+| Document | Path | Processed? | Summary At |
+|----------|------|-----------|------------|
+| [name]   | `[path]` | Yes/No | `[summary path]` |
+
+## Success Criteria
+- [criterion 1]
+- [criterion 2]
+
+## Known Constraints
+- [constraint 1]
+- [constraint 2]
+
+## Project Phase Tracker
+| Phase | Status | Summary File | Date |
+|-------|--------|-------------|------|
+| Discovery | Not Started / In Progress / Complete | `[path]` | |
+| Strategy | Not Started / In Progress / Complete | `[path]` | |
+| Deliverable Draft | Not Started / In Progress / Complete | `[path]` | |
+| Review & Polish | Not Started / In Progress / Complete | `[path]` | |
+```
+
+---
+
+## Template 6: Task Definition
+
+**Use when:** Defining a discrete unit of work before starting execution
+
+```markdown
+## Task: [name]
+**Date:** [YYYY-MM-DD]
+**Client:** [if applicable]
+**Work Type:** [proposal / workshop / analysis / content / agent development]
+
+### Context Files to Load
+- `[file path]` — [why needed]
+
+### Action
+[What to produce. Be specific about format, length, and scope.]
+
+### Verify
+- [ ] Numbers match source data exactly
+- [ ] Open questions marked OPEN
+- [ ] Output matches what was requested, not what was assumed
+- [ ] Claims backed by specific data
+- [ ] Consistent with stored decisions in docs/context/
+
+### Done When
+- [ ] Output file exists at `[specific path]`
+- [ ] Summary written to `docs/summaries/[specific file]`
+```
+
+---
+
+## Subagent Output Contracts
+
+**CRITICAL: When subagents return results to the main agent, unstructured prose causes information loss. These output contracts define the EXACT format subagents must return.**
+
+### Contract for Document Analysis Subagent
+
+```
+=== DOCUMENT ANALYSIS OUTPUT ===
+SOURCE: [file path]
+TYPE: [document type]
+CONFIDENCE: [high/medium/low]
+
+NUMBERS:
+- [metric]: [exact value]
+[repeat for all numbers found]
+
+REQUIREMENTS:
+- REQ: [requirement] | CONDITION: [if any] | PRIORITY: [level] | CONSTRAINT: [if any]
+[repeat]
+
+DECISIONS_REFERENCED:
+- DEC: [what] | WHY: [rationale] | BY: [who]
+[repeat]
+
+CONTRADICTIONS:
+- [this document says X] CONTRADICTS [other known fact Y]
+[repeat or NONE]
+
+OPEN:
+- [unresolved item] | NEEDS: [who/what to resolve]
+[repeat or NONE]
+
+QUOTES:
+- "[verbatim]" — [speaker], [context]
+[repeat, max 5]
+
+=== END OUTPUT ===
+```
+
+### Contract for Research/Analysis Subagent
+
+```
+=== RESEARCH OUTPUT ===
+QUERY: [what was researched]
+SOURCES: [list]
+CONFIDENCE: [high/medium/low] BECAUSE [reason]
+
+CORE_FINDING: [one sentence]
+
+EVIDENCE:
+- [data point]: [exact value] | SOURCE: [where] | DATE: [when]
+[repeat]
+
+CONCLUSIONS:
+- IF [condition] THEN [conclusion] | EVIDENCE: [reference]
+[repeat]
+
+GAPS:
+- [what was not found or not covered]
+[repeat or NONE]
+
+NEXT_STEPS:
+- [recommended action] | PRIORITY: [level]
+[repeat]
+
+=== END OUTPUT ===
+```
+
+### Contract for Review/QA Subagent
+
+```
+=== REVIEW OUTPUT ===
+REVIEWED: [file path or deliverable name]
+AGAINST: [what standard — spec, requirements, style guide]
+
+PASS: [yes/no/partial]
+
+ISSUES:
+- SEVERITY: [critical/major/minor] | ITEM: [description] | LOCATION: [where in document] | FIX: [suggested resolution]
+[repeat]
+
+MISSING:
+- [expected content/section not found] | REQUIRED_BY: [which requirement]
+[repeat or NONE]
+
+INCONSISTENCIES:
+- [item A says X] BUT [item B says Y] | RESOLUTION: [suggested]
+[repeat or NONE]
+
+STRENGTHS:
+- [what works well — for positive reinforcement in iteration]
+[max 3]
+
+=== END OUTPUT ===
+```
+
+---
+
+## Phase-Based Workflow Templates
+
+### Template A: Enterprise Sales Deliverable
+
+```
+Phase 1: Discovery & Input Processing
+├── Process all client documents → Source Document Summaries
+├── Identify gaps in information → flag as OPEN items
+├── Create Decision Records for any choices made
+├── Write: ./docs/summaries/01-discovery-complete.md (Handoff Template)
+├── → Suggest new session for Phase 2
+
+Phase 2: Strategy & Positioning
+├── Read summaries only (NOT source documents)
+├── Competitive positioning analysis → Analysis Summary
+├── Value proposition development
+├── ROI framework construction with EXACT numbers
+├── Write: ./docs/summaries/02-strategy-complete.md (Handoff Template)
+├── → Suggest new session for Phase 3
+
+Phase 3: Deliverable Creation
+├── Read strategy summary + project brief only
+├── Draft deliverable (proposal / deck / workshop plan)
+├── Output to: ./output/deliverables/
+├── Write: ./docs/summaries/03-deliverable-draft.md (Handoff Template)
+├── → Suggest new session for Phase 4
+
+Phase 4: Review & Polish
+├── Read draft deliverable + strategy summary
+├── Quality review using Review/QA Output Contract
+├── Final edits and formatting
+├── Output final version to: ./output/deliverables/
+```
+
+### Template B: Agent/Application Development
+
+```
+Phase 1: Requirements → Spec
+├── Process all input documents → Source Document Summaries
+├── Generate structured specification
+├── Output: ./output/SPEC.md
+├── Write: ./docs/summaries/01-spec-complete.md (Handoff Template)
+├── → Suggest new session for Phase 2
+
+Phase 2: Architecture → Schema
+├── Read SPEC.md + summaries only
+├── Design data model
+├── Define agent behaviors and workflows
+├── Output: ./output/schemas/data-model.yaml
+├── Output: ./output/schemas/agent-definitions.yaml
+├── Write: ./docs/summaries/02-architecture-complete.md (Handoff Template)
+├── → Suggest new session for Phase 3
+
+Phase 3: Prompts → Integration
+├── Read schemas + spec only
+├── Write system prompts for each agent
+├── Map API integrations and data flows
+├── Output: ./output/prompts/[agent-name].md (one per agent)
+├── Output: ./output/schemas/integration-map.yaml
+├── Write: ./docs/summaries/03-prompts-complete.md (Handoff Template)
+├── → Suggest new session for Phase 4
+
+Phase 4: Assembly → Package
+├── Read all output files
+├── Assemble complete application package
+├── Generate deployment/setup instructions
+├── Output: ./output/deliverables/[project]-complete-package/
+├── QA check against original spec using Review/QA Output Contract
+```
+
+### Template C: Hybrid (Sales + Agent Development)
+
+```
+Phase 1: Client Discovery → Summaries
+Phase 2: Solution Design → Architecture + Schema
+Phase 3a: Client-Facing Deliverable (proposal/deck)
+Phase 3b: Internal Technical Package (schemas/prompts)
+Phase 4: Review both tracks against each other for consistency
+```
+
+---
+
+## End of Templates
+
+**Return to your task after reading the template(s) you need. Do not keep this file in active context.**

vault-id-from-op-client.sh

@@ -21,9 +21,23 @@ if [[ -z "$VAULT_ID" ]]; then
     exit 1
 fi
 
+# Skip silently for the default vault ID (no named vault to look up)
+if [[ "$VAULT_ID" == "default" ]]; then
+    echo "default"
+    exit 0
+fi
+
 ITEM_NAME="${VAULT_ID} vault key"
 FIELD_NAME="password"
 
+# Skip silently if 1Password is not available or not authenticated
+if ! command -v op &>/dev/null; then
+    exit 0
+fi
+if [[ -z "$OP_SERVICE_ACCOUNT_TOKEN" && -z "$OP_CONNECT_HOST" && ! -S "${HOME}/.1password/agent.sock" ]]; then
+    exit 0
+fi
+
 # Fetch the vault password from 1Password
 VAULT_PASSWORD=$(op item get "$ITEM_NAME" --fields "$FIELD_NAME" --format=json  --vault LabSecrets 2>/dev/null | jq -r '.value')