# Operator Portal — Plan `operator.dezky.local` (dev) → `operator.dezky.com` (prod). Internal admin portal for Dezky staff: managing tenants, partners, operating the platform. Distinct from the customer portal at `app.dezky.local`. Different OAuth client, different cookie domain, different surface — though they share Authentik as the IdP and (eventually) platform-api as the backend. This file is the running record of decisions made during the design grilling session. Updated inline as questions resolve. --- ## Scope — C-visual with real management for Tenants + Partners Decision: build every screen from the source design visually, but back two domains with real CRUD from day one — Tenants and Partners. Everything else renders against mock-data fixtures until its backend is built. | Surface | Day-1 state | |---|---| | Overview / dashboard | Visual — aggregates from real Tenant+Partner data where available, mock for the rest | | Tenants (list + detail with 7 tabs) | **Real backend**, full CRUD, suspend/resume/delete | | Partners (list + detail) | **Real backend**, new schema, full CRUD | | Users (global) | Real read across tenants (already in DB) | | Support queue | Mock | | Platform billing | Mock | | Reports | Mock | | Infrastructure | Visual; could derive from Docker health checks but probably mock initially | | Feature flags | Mock | | Audit log | Mock (real backfill is a follow-up) | | Operator team | Real (Users with `platformAdmin: true`) | | Platform settings | Mock | | Command palette ⌘K | Visual — opens, navigates, but "execute action" just toasts | | Impersonation modal + banner | Visual — confirms the action but doesn't actually mint a token | | Incident modal | Mock | | Env switcher (prod/staging/dev) | Cosmetic — picks a label, no real env switch | | On-call indicator | Mock | ### Real-backend surface this adds Two genuinely new things on the backend: 1. **Partner schema and CRUD** in `services/platform-api` — id, name, domain, status, customers count (computed), MRR (computed), margin, sinceDate. Tenants gain an optional `partnerId` field. The existing `dezky` seed gets no partner. 2. **Tenant lifecycle actions** beyond create — suspend, resume, change plan, change seat cap, soft-delete with grace period. Existing schema covers most of this; controllers need new methods. Everything else (incidents, flags, support tickets, audit log collection, impersonation tokens) stays mock until explicitly promoted. --- ## Lives at `apps/operator/` — separate Nuxt app Decision: new Nuxt 3 app, separate `package.json`, separate Traefik route at `operator.dezky.local`. Reuses design tokens / NodeMark / UiIcon by copy for now; a `packages/ui` workspace is a likely follow-up once we have a third consumer. **Why separate, not a route group in `apps/portal/`:** security boundary. The moment any operator-only feature mutates customer state (impersonation, suspend tenant), a routing or middleware bug on a shared app is catastrophic. Separate apps make that nearly impossible. Different cookies, different OIDC client, different domain. **Cost:** one more docker-compose service, ~10 lines of Traefik labels, one more volume for `node_modules`. Some duplicated dev tooling (eslint, tsconfig). --- ## Auth — new `dezky-operator` Authentik OAuth provider Decision: a dedicated OAuth client in Authentik, distinct from `dezky-portal`. - New provider `dezky-operator` (confidential, PKCE on) - Redirect URIs: `https://operator.dezky.local/auth/oidc/callback` - Group binding: `dezky-platform-admins` required at the provider's authorization flow (Authentik policy), so non-admins can't even consent - Stricter policies attached only to this provider: MFA required, future IP allowlist for the office network/VPN - Token audience claim: `dezky-operator` - Provisioning's `JwtAuthGuard` widens its audience check to a list: `['dezky-portal', 'dezky-operator']` - Per-endpoint guard for operator-only mutations: require `aud === 'dezky-operator'` AND `actor.platformAdmin === true`. The audience check makes "is this a privileged session" provable from the token alone, independent of the DB lookup **UX trade-off accepted:** if Ronni (or any operator who is also a customer) wants to be in both apps, they log into Authentik twice — once per audience. Correct security-wise, fine ergonomically. --- ## Backend stays as one service — rename to `services/platform-api` Decision: route all operator mutations and reads through the existing NestJS service (no second backend, no Nitro-direct-to-Mongo). Rename `services/provisioning` → `services/platform-api` because the service now owns more than just provisioning — it's the platform's data + control plane. **What changes during the rename:** - Directory: `services/provisioning/` → `services/platform-api/` - Package: `@dezky/provisioning` → `@dezky/platform-api` - Docker container name: `dezky-provisioning` → `dezky-platform-api` - Compose service key, network alias, volume names - Portal env var: `PROVISIONING_INTERNAL_URL` → `PLATFORM_API_INTERNAL_URL` - Portal proxy routes: `http://provisioning:3001` → `http://platform-api:3001` - Internal module names referencing "provisioning" stay (e.g. `ProvisioningService` is now one orchestration concern *inside* `platform-api`; not the whole service's purpose) - Public URL stays `api.dezky.local` (Traefik routes by Host header, unaffected) **New endpoints platform-api gains in this phase:** - `POST /tenants/:slug/suspend`, `POST /tenants/:slug/resume` - `PATCH /tenants/:slug` already exists; ensure it can change plan / seat cap - `GET /partners`, `POST /partners`, `GET /partners/:slug`, `PATCH /partners/:slug` - `Tenant.partnerId` foreign key + filter on tenant queries - `JwtAuthGuard` accepts both `dezky-portal` and `dezky-operator` audiences; per-endpoint requirement of `dezky-operator` aud for operator-only mutations **Strategy:** rename in a separate prep commit before the operator work starts, so the rename diff is mechanical and reviewable on its own. --- ## Partner schema ```typescript @Schema({ collection: 'partners', timestamps: true }) class Partner { slug: string // 'nordicmsp', URL-safe, unique name: string // 'NordicMSP' domain: string // 'nordicmsp.dk' — partner's own org domain status: 'active' | 'in-negotiation' | 'paused' | 'terminated' // default 'in-negotiation' marginPct: number // 20 = partner keeps 20% of customer MRR; one number per partner partnershipStartedAt?: Date contactInfo: { primaryName?, primaryEmail?, billingEmail? } billingInfo: { /* same shape as Tenant.billingInfo */ } } ``` **Tenant side:** add `partnerId?: Types.ObjectId` (ref Partner, indexed, optional). Direct customers have no `partnerId`; partner-owned customers reference one. **Computed at query time, not stored:** - `Partner.customers` — count of tenants with `partnerId === this._id` - `Partner.mrr` — sum of those tenants' MRR Storing denormalized would force write-time syncing on every tenant create/suspend/plan-change for ~zero benefit at our scale. **Operator-only.** A self-serve partner portal at `partner.dezky.local` is a future surface; not in this phase. Partners are visible/manageable only from the operator app. --- ## Impersonation — visual stub now, real flow later Decision: build the UI exactly as designed (modal with reason field, top banner, exit button) but do not wire actual token exchange. The confirm action toasts "impersonation not implemented yet" and writes a mock audit entry. **Why now:** validates the UX, lets future hires see the operator surface end-to-end, doesn't introduce a dangerous capability before there's an operational need. **Mitigations against confusion:** - Modal carries a `Demo only` badge — same styling as other stub-data badges in the operator UI - Toast on confirm makes the no-op explicit - The banner does display in mock mode (so we can iterate on its design), but the underlying session state is local to the operator tab **Real flow design recorded for the follow-up:** OAuth 2 Token Exchange (RFC 8693). Authentik supports it. Customer portal needs to accept tokens carrying an `act` claim alongside `sub`, and show its own impersonation banner when the two differ. ~2 days of careful work + security review. --- ## Decisions made without grilling (small, low-risk) - **Theme:** dark by default. Existing `apps/portal/assets/styles/tokens.css` already defines `[data-theme='dark']` tokens; the operator app sets `` at app root and reuses them - **Mock data location:** TypeScript files under `apps/operator/data/` (`tenants-mock.ts`, `partners-mock.ts`, `flags-mock.ts`, etc.). Same shape as `operator-data.jsx` from the design bundle, just retyped - **Design system reuse:** copy `NodeMark.vue`, `UiIcon.vue`, and the auth components into `apps/operator/components/` directly. A shared `packages/ui` workspace becomes worth doing once a third surface needs them (partner portal? landing site?) - **OCIS / Stalwart admin shortcuts in operator UI:** out of scope for this phase. Operator drills via the customer-facing service URLs --- ## Follow-up tasks (post-MVP) In rough priority order: 1. **Real impersonation flow** — OAuth Token Exchange (RFC 8693), customer portal `act`-claim handling, audit on entry+exit, banner with origin operator identity 2. **Real audit log collection** — replace mock fixtures with a `platform_audit` collection in Mongo that platform-api writes on every privileged action; stream from there in the operator UI 3. **Feature flag backend** — `Flag` schema + per-tenant rollout state + a tiny flag-eval client every service imports 4. **Incident management backend** — `Incident` schema + paging integration (PagerDuty / OpsGenie / custom). Until then, the incident modal renders from mock 5. **Support ticket queue** — `SupportTicket` schema + email-in ingestion from a dedicated mailbox via Stalwart 6. **Self-serve Partner portal at `partner.dezky.local`** — Phase 6+ work, own Nuxt app, own OAuth client, scoped to a partner's own customers 7. **Real environment switcher** — currently cosmetic; would need separate API endpoints per env, separate Authentik tenants, etc. 8. **Real on-call indicator** — integration with the paging system that gets installed in (4) 9. **Operator workspace impersonation in OCIS/Stalwart** — operator tooling reaches *into* the customer's file storage and mail for support, with the same audit trail as portal impersonation --- ## Out of scope for this entire effort - Multi-region operator UI - Read-only investor / board mode (a real persona but build it when there's a real investor — design has a placeholder "Read-only" role for Jonas Berg) - White-label of the operator portal (partners get their own portal eventually; Dezky operator never gets white-labeled — it's our internal tool) --- ## Execution checklist Tick boxes as work lands. Each phase is roughly one commit. Phases must be done in order — earlier ones unblock later ones. ### O.0 · Prep — service rename ✓ - [x] Rename `services/provisioning/` → `services/platform-api/` - [x] Update `package.json` name → `@dezky/platform-api` - [x] Update `docker-compose.yml`: container name, service key, volume name, env var `PROVISIONING_INTERNAL_URL` → `PLATFORM_API_INTERNAL_URL`, NUXT_API_BASE points at new hostname - [x] Update portal proxy routes to read `PLATFORM_API_INTERNAL_URL` and default to `http://platform-api:3001` - [x] Sweep docs (README, CLAUDE.md, SERVICES.md, AUTHENTIK-SETUP.md, NEXT-STEPS.md, TROUBLESHOOTING.md) for stale references - [x] Verify customer portal `/api/me` still works end-to-end after rename ### O.1 · Authentik — operator OAuth client ✓ - [x] Create `dezky-operator` OAuth provider via Authentik API - [x] Set redirect URIs to `https://operator.dezky.local/auth/oidc/{callback,logout}` - [x] Confidential client; client_secret persisted to `.env` as `OPERATOR_OIDC_CLIENT_SECRET` - [x] `Dezky Operator` application created and linked to the provider - [x] Group binding on the application: `dezky-platform-admins` required to reach the consent screen. (Authentik 2025.10 supports group-direct policy bindings — no separate `policy_group_membership` object needed) - [ ] **Deferred to follow-up:** MFA-required policy on this provider. Authentik does this via a stage binding on the authentication flow, which is app-specific configuration we'll wire when there's an actual MFA enrollment to gate against. For dev with one akadmin, akadmin already has WebAuthn — the auth flow prompts for it automatically - [x] Discovery doc verified at `/application/o/dezky-operator/.well-known/openid-configuration` — issuer correct, scopes include `groups`, all endpoints resolve ### Gotchas worth noting - Authentik 2025.10 requires both `authorization_flow` AND `invalidation_flow` when creating OAuth2 providers. The default invalidation flow is at `/api/v3/flows/instances/?designation=invalidation` (slug `default-provider-invalidation-flow`) - The `policies/group_membership/` endpoint mentioned in older Authentik docs is gone in 2025.10. Use `policies/bindings/` with a direct `group` reference instead ### O.2 · platform-api — multi-audience + Partner CRUD ✓ - [x] `JwtAuthGuard`: accepts comma-separated `AUTHENTIK_AUDIENCE` (`dezky-portal,dezky-operator`). Both audiences validate; per-endpoint guards further restrict - [x] `OperatorGuard` (not a decorator — a regular `CanActivate` guard) enforcing `aud includes 'dezky-operator' && actor.platformAdmin`. Applied via `@UseGuards(JwtAuthGuard, OperatorGuard)` - [x] `schemas/partner.schema.ts` — Partner model - [x] `partners/` module: controller + service + DTOs (create / read / update / soft-terminate / list tenants under partner) - [x] `partnerId?: Types.ObjectId` added to Tenant schema (indexed, sparse). `UpdateTenantDto` accepts `partnerId` to attach/detach - [x] `Partner.customers` aggregated at query time (count of Tenants by partnerId). MRR aggregation **deferred** — Tenant has no monthly amount yet and Subscription lacks a price column. Will land when Subscription gains pricing - [x] Tenant lifecycle endpoints: `POST /tenants/:slug/suspend`, `POST /tenants/:slug/resume` (operator-only). PATCH already accepts plan/domains/partnerId changes - [x] Smoke test: customer-portal token → `POST /partners` returns 403 "This endpoint requires an operator-scoped token" ✓. Positive test (operator token → 200) deferred until O.3 when the operator app exists to mint that token ### O.3 · Scaffold `apps/operator/` ✓ - [x] `apps/operator/package.json` (Nuxt 3, `nuxt-oidc-auth` 1.0.0-beta.11) - [x] `nuxt.config.ts` wired against the `dezky-operator` Authentik provider: `client_id=dezky-operator`, audience claim becomes `dezky-operator`, scope includes `groups`, `exposeAccessToken: true` so the Nitro proxy can forward it - [x] Docker compose service `operator` running on the dezky network, mkcert root CA mounted, Traefik route at `operator.dezky.local` - [x] Network alias on Traefik: `operator.dezky.local` - [x] `operator.dezky.local` added to `/etc/hosts` - [x] Distinct session secrets in `.env` (`OPERATOR_NUXT_OIDC_*`) — the two apps can't decrypt each other's session cookies - [x] Verified login: signing in lands on the placeholder index showing `Operator portal · placeholder` with the user's identity - [x] Smoke test `POST /partners`: operator session returns 200 (partner created in Mongo), idempotent re-call returns 409 (already exists), customer-portal session returns 403 ("requires operator-scoped token") - [x] `JwtAuthGuard` extended to accept **multi-issuer** as well as multi-audience (each Authentik OAuth provider has its own per-app `iss` URL); `AUTHENTIK_ISSUER` env is now comma-separated. The audience change in O.2 wasn't enough on its own — issuer matching is separate ### O.4 · Design system + app shell ✓ - [x] `assets/styles/tokens.css` carbon-default (done in O.3) - [x] `assets/styles/base.css` (done in O.3) - [x] `NodeMark.vue` (copied unchanged from portal), `UiIcon.vue` (expanded set: 31 icons covering sidebar/topbar/sort/arrows) - [x] Shared primitives: `Card`, `UiButton` (5 variants × 3 sizes), `DataTable`, `Badge` (7 tones), `Mono`, `Eyebrow`, `StatusDot`, `Avatar` (deterministic palette), `PageHeader` - [x] `OpSidebar.vue` — collapsible (232↔56px), 12 nav items in 4 sections, active-row highlight from route, badge slot per item, brand mark + user identity footer - [x] `OpTopbar.vue` — env badge (prod/staging/dev), ⌘K palette trigger stub, on-call pill, bell, avatar - [x] `layouts/default.vue` wires sidebar + topbar + ``; `layouts/blank.vue` for the login page; `app.vue` uses `` - [x] Keyboard shortcut: ⌘[ collapses/expands sidebar (verified — width flips 232↔56 in the browser via the toggle click). ⌘K palette lands in O.8 - [x] Verified in browser: shell renders with all 12 nav links, env badge shows PROD, PageHeader title resolves to the user's display name, smoke test re-confirmed 409 on the seeded `test-partner` (token forwarding still works after the layout refactor) ### Gotcha worth noting - Vite 7.3 added a strict `server.allowedHosts` check that blocks any Host header that isn't an exact match for the dev origin. The customer portal was scaffolded under an older Vite and pre-dates this. Operator needs `allowedHosts: ['operator.dezky.local']` in `nuxt.config.ts` under `vite.server` or every request 403s with a plaintext error. ### O.5 · Tenant management (real backend) ✓ - [x] `pages/tenants/index.vue` — list with status/plan/domains/created/ provisioning-state columns; search by slug or name; status chips (all / active / pending / suspended) with live counts; click-through to detail - [x] `pages/tenants/[slug].vue` — detail with 7 tabs (`Tabs` primitive) - [x] Tab: **Overview** — Identity card + Billing card with key fields - [x] Tab: **Users** — list users via `GET /api/tenants/:slug/users` (new endpoint added to platform-api), lazy-loaded on first tab click - [x] Tab: **Resources** — per-integration `Badge` + external handle (group ID / mail domain / space ID) + last error if any; **Reconcile now** button re-runs orchestration in place. Iterates the explicit `INTEGRATIONS` array, not the raw Mongoose subdoc keys, so the `_id` field doesn't leak into the UI - [x] Tab: **Billing** (mock — plan + MRR + Stripe IDs as fixtures) - [x] Tab: **Audit** (mock — three sample log entries) - [x] Tab: **Support** (mock — placeholder) - [x] Tab: **Danger zone** — three real-backend cards (Suspend / Resume / Soft-delete) each gated by ConfirmDialog. Suspended verified live: acme → `suspended` in Mongo, then Resumed back to `active` ### Server proxies added (apps/operator/server/api/tenants/) `platform-api.ts` util encapsulates the access-token forwarding. Routes: `index.get`, `[slug]/index.{get,patch,delete}`, `[slug]/users.get`, `[slug]/{suspend,resume,reconcile}.post`. All read the operator's session, forward as bearer to platform-api. ### New primitives this phase `Tabs.vue` (horizontal strip with optional counts), `ConfirmDialog.vue` (Teleport-to-body modal, Escape/backdrop close, danger/primary tone). ### Gotchas worth noting - **Mongoose subdoc `_id` leaks into JSON**: iterating `v-for="(state, k) in tenant.provisioningStatus"` includes `_id`. Iterate an explicit whitelist (`['authentik', 'stalwart', 'ocis']`) instead. - **Fields added to schema after document creation are missing in old docs**: acme was created before `provisioningErrors` existed, so `tenant.provisioningErrors[k]` throws `Cannot read properties of undefined`. Use optional chaining (`tenant.provisioningErrors?.[k]`). - Nitro can throw `Could not load /app/server/api/...` if Vite picks up a new file mid-build. Container restart clears it. ### O.6 · Partner management (real backend) ✓ - [x] `pages/partners/index.vue` — list with name/domain/status/customers/margin - [x] `pages/partners/[slug].vue` — detail panel with contract card, contact card, customers table, attach modal, terminate danger card - [x] "Create partner" modal — POST /partners, navigates to detail on success - [x] Attach / detach tenant to partner (PATCH on tenant.partnerId, with $unset for detach so the field disappears cleanly) - [x] `services/platform-api/src/schemas/tenant.schema.ts` — added the `partnerId` Prop. It was missing, which is why early PATCH attempts returned 200 but Mongoose silently dropped the field. Smoke-tested with acme ⇄ nordicmsp and a throwaway temp-msp partner (created + terminated). - MRR aggregation deferred until Subscription gains real pricing (see follow-ups). For now `customers` is just a count of attached tenants. ### O.7 · Visual-only screens (mock fixtures) ✓ - [x] `data/fixtures.ts` — typed mock fixtures (SERVICES, INCIDENT, FLAGS, OP_AUDIT). Tenant/partner/user extras are NOT mocked — those screens pull from the real backend. - [x] `pages/index.vue` — Overview dashboard: KPIs from real tenants/partners /users + status meter + recent + needs-follow-up tables, with mock activity stream + incident banner overlay. - [x] `pages/operator-team.vue` — real `GET /users` filtered to `platformAdmin === true`. - [x] `pages/users.vue` — real `GET /users` with All / Admins / Inactive views and search. - [x] `pages/infrastructure.vue` — service health (mock SERVICES); docker healthcheck + Prometheus wiring is a follow-up. - [x] `pages/flags.vue` — feature flags (mock FLAGS). - [x] `pages/audit.vue` — cross-tenant audit (mock OP_AUDIT) with search. - [x] `pages/support.vue` — `OpPlaceholder` stub. - [x] `pages/billing.vue` — `OpPlaceholder` stub. - [x] `pages/reports.vue` — `OpPlaceholder` stub. - [x] `pages/settings.vue` — `OpPlaceholder` stub. - [x] Shared bits added: `components/Stat.vue`, `components/MetricCell.vue`, `components/OpPlaceholder.vue`, `server/api/users/index.get.ts`, `types/user.ts`. ### O.8 · Interactions ✓ - [x] `CommandPalette.vue` + `useCommandPalette` — ⌘K opens, searches real tenants/partners + mock flags + nav items + actions. Arrow keys + Enter navigate, Escape/backdrop close. - [x] `ImpersonationModal.vue` + `useImpersonation` — confirm modal with reason field, opens from tenant detail (`Impersonate` action) and from the palette (`Impersonate user…` action). Stub — no real OBO token is minted. Follow-up to wire OAuth Token Exchange remains. - [x] `ImpersonationBanner.vue` — full-width red banner at the top of the shell, persists until `Exit impersonation` is clicked. - [x] `IncidentModal.vue` + `useIncidentModal` — opens from the Overview and Infrastructure incident banners, renders mock `INCIDENT` data (metrics + timeline + draft composer). - [x] `TweaksPanel.vue` + `useTweaks` — floating bottom-right panel. Theme (dark/light), density (comfy/compact), env badge (prod/staging/dev). Choices persist to localStorage, apply via `[data-theme]` / `[data-density]` overrides in tokens.css. - [x] Layout wires ⌘K + ⌘[ globally. Topbar reads env from `useTweaks`. ### O.9 · Verification ✓ All smokes ran end-to-end on 2026-05-24 against the live local stack. - [x] Signed in to `operator.dezky.local` as akadmin via the `dezky-operator` OAuth client. - [x] JWT audience confirmed via `GET /api/_verify-token`: ``` iss: https://auth.dezky.local/application/o/dezky-operator/ aud: dezky-operator sub: bc865e33... groups: [authentik Admins, dezky, dezky-platform-admins] ``` - [x] Created `verify-msp` Partner via the UI ("New partner" modal) — verified in Mongo: `_id: 6a129d6a44c0f44fddda34bf`, `marginPct: 15`. - [x] Attached `acme` tenant via the Attach modal on the partner detail page; Mongo confirmed `tenants.acme.partnerId == partners.verify-msp._id` and the customers count in the UI rose from 0 → 1. - [x] Suspended `acme` from the Danger tab — Mongo confirmed `tenants.acme.status == 'suspended'`. Resumed it back to `active` afterwards so the dev tenant stays usable. - [x] Signed in to `app.dezky.local` in a parallel tab; `GET /api/_verify-token` there returned `aud: dezky-portal`, `iss: .../dezky-portal/`. Both sessions coexist; each app uses its own per-app issuer + audience. `GET /api/me` on the portal still returns profile + tenants + subscriptions correctly. - [x] Follow-up tasks rolled into NEXT-STEPS.md under "Follow-ups before operator hits production". ### Throwaway artifacts left behind for now These were added during O.9 verification and can be ripped out when the relevant production gates land: - `apps/operator/server/api/_verify-token.get.ts` — JWT claim echo - `apps/portal/server/api/_verify-token.get.ts` — JWT claim echo - `apps/operator/server/api/operator-smoke-test.post.ts` — O.3-era audience check - `apps/portal/server/api/partners/index.post.ts` — O.2-era audience-deny verifier