A root-run z-push-admin (kubectl exec defaults to root) left a
root-owned 'users' file on the state PVC; Apache runs as www-data, so
every request 500'd with 'Not possible to write to the configured
state directory'. An initContainer now normalizes ownership on every
start (state is disposable, ownership isn't precious), and the docs
say to exec z-push-admin as www-data.
BackendCombined login is all-or-nothing, and Stalwart returns 404 for
/dav/card/<account>/ when the account's default address book was never
created (it doesn't auto-create on the gateway's PROPFIND the way the
calendar home worked) — so CardDAV killed every otherwise-successful
EAS login. Exchange accounts now bundle mail + calendar; contacts stay
on the Apple-profile CardDAV path. Re-enable BackendCardDAV once
platform-api provisions a default address book at mailbox creation.
Copy/docs aligned: portal hint, SERVICES.md, website FAQ (da+en).
Stalwart never had ActiveSync built in — that now comes from the
zpush gateway. SERVICES.md gains a zpush section with debug
commands; website copy (da+en) states what actually works: IMAP
everywhere, CalDAV/CardDAV, Exchange accounts in the phone's
built-in apps, CalDAV Synchronizer for Outlook on Windows.
- Request offline_access for the ocis-web client (WEB_OIDC_SCOPE) so the web
SPA gets a refresh token and renews silently instead of dropping the session
(no surprise logouts; the "no permission to upload" symptom was the
expired-token state). The ocis-provider already has the offline_access scope
mapping; its access-token validity is bumped 5m → 1h (refresh 30d).
- Flatten the remaining brand gradients in index.html: the active sidebar
highlight (.oc-background-primary-gradient) and primary buttons
(.oc-button-primary-filled) are now solid carbon (text stays light/readable).
- Document the offline_access + token-validity provider settings in
AUTHENTIK-SETUP.md (the provider lives in Authentik's DB, not git).
The Storage page + endpoint landed earlier but had no working OCIS
backend credential. OCIS has no service-account/client-credentials grant
and trusts a single issuer, and basic auth resolves no user in our
external-IdP setup — so authenticate OcisClient via an OIDC
refresh-token bootstrap instead:
- One-time headless login of svc-platform-api against the ocis provider
(public client ocis-web, issuer .../o/ocis/) yields a refresh token,
persisted in Mongo (ocis_credentials) and rotated on every use.
- OcisClient mints access tokens with the refresh_token grant; the
service user holds the OCIS admin role (OCIS_ADMIN_USER_ID) so
libregraph ListAllDrives works.
- scripts/bootstrap-ocis.mjs re-runs the bootstrap if the token lapses.
- Dashboard Plan card gains a storage capacity bar beside seats;
hidden when storage is unavailable.
- compose + .env.example: OCIS service OIDC env and admin user id.
- docs/NEXT-STEPS: document the mechanism and the dead-end alternatives.
A partner or tenant admin could complete the dezky-operator OIDC flow and
land on the operator portal. The platform-api OperatorGuard already 403s
their data, but the login/UI layer had no authorization check at all — the
only gate was a manual Authentik UI setting with nothing in git enforcing it.
Close it with defense-in-depth across three independent layers:
1. IdP — operator-application.yaml blueprint binds an
ak_is_group_member("dezky-platform-admins") policy to the dezky-operator
app, so Authentik denies the OIDC flow for non-admins. The blueprint also
provisions the provider + application (state: created, so a fresh env is
built from code while an existing hand-made provider is left untouched).
Wire OPERATOR_OIDC_* into both authentik containers and mount the
blueprints dir on the worker (it applies blueprints, and previously lacked
the mount).
2. Operator app — require-platform-admin.global.ts requires platformAdmin and
routes a non-admin to not-authorized.vue, which triggers a full sign-out
(local + Authentik IdP) for shared-workstation safety. Fails open on a
transient /api/me error by design, to avoid mass-signout on platform-api
restarts; layers 1 and 3 contain the exposure.
3. platform-api — OperatorGuard (unchanged) requires dezky-operator audience
plus platformAdmin resolved from the DB on every request.
Also harden the partner surface: it shares the dezky-portal client with tenant
users so it has no IdP gate, and its /partner/* route middleware now fails
CLOSED when identity can't be confirmed.
Docs (AUTHENTIK-SETUP.md) and .env.example updated; the operator client secret
must be set before first boot since the blueprint now consumes it.
New docs/FEATURE-FLAGS.md captures when to add a flag, where the moving
parts live, how to use useFeatureFlag from app code, the 4 states + 4
scope axes, kill-switch flow, naming conventions, and the parts we know
aren't built yet (partnerSlug eval context, user-level flags, audit-log
integration, server-side cache).
CLAUDE.md gets a one-line convention entry under "Code conventions" so
future devs notice it when grepping for code rules. NEXT-STEPS.md is
updated: the feature-flag backend follow-up is now ticked done with a
pointer to FEATURE-FLAGS.md for the remaining sub-tasks, and the
"What landed" section reflects the real Infrastructure + Flags pages
and the notification drawer.
- Add _verify-token.get.ts to both operator and portal — decodes the
access token stored in the nuxt-oidc-auth session and echoes iss/aud/
sub/groups. Used to confirm operator tokens carry aud=dezky-operator
and portal tokens carry aud=dezky-portal. Listed in NEXT-STEPS.md as
throwaway, to be removed when proper verification surfaces exist.
- OPERATOR-PLAN.md O.9 marked done with the actual claims captured + the
Mongo-side verification of attach + suspend flows.
- NEXT-STEPS.md: replaced the "Operator portal — out-of-band track"
section with a "shipped + follow-ups" version. The 9-item follow-up
list (impersonation, audit, flags, incidents, support, partner
portal, env switcher, on-call, workspace impersonation) is now the
authoritative roadmap, not buried inside OPERATOR-PLAN.md.
- CommandPalette + useCommandPalette: ⌘K opens a search-and-jump panel over
real tenants/partners + fixture flags + nav + actions. Arrow keys + Enter
navigate, Escape/backdrop close. Recents are intentionally omitted for now;
add when there's something to recent over.
- Impersonation stub: useImpersonation + ImpersonationModal + ImpersonationBanner.
Modal opens from tenant detail and from the palette. Banner stays at the top
of the shell until exited. No real OBO token is minted — wiring OAuth Token
Exchange is tracked as a follow-up.
- IncidentModal + useIncidentModal: opened from the Overview and Infrastructure
incident banners, renders the mock INCIDENT data with metrics, timeline and
draft composer.
- TweaksPanel + useTweaks: floating bottom-right panel for theme (dark/light),
density (comfy/compact), env badge (prod/staging/dev). Saved to localStorage.
- Theme/density apply via [data-theme] + [data-density] overrides in
tokens.css. Topbar env badge now reads from useTweaks instead of a prop.
- Layout wires ⌘K + ⌘[ at the document level and mounts the palette + modals
+ banner + tweaks panel once for all pages.
- Overview (pages/index.vue): KPIs from real /tenants /partners /users,
status meter, recent + needs-follow-up tables. Mock activity stream and
incident banner overlay come from data/fixtures.ts.
- Operator team: real GET /users filtered to platformAdmin === true,
with last-seen + tenant counts.
- Users (global): real read with All/Admins/Inactive views and search.
- Infrastructure / Feature flags / Audit: mock fixtures only — wiring to
real backends (Prometheus, OpenFeature, append-only audit) is tracked
as follow-ups in OPERATOR-PLAN.md.
- Placeholder pages (support/billing/reports/settings) via OpPlaceholder.
- Shared: Stat, MetricCell, OpPlaceholder components, /api/users proxy,
PlatformUser type.
- .gitignore: scope the docker volumes data/ rule so apps/*/data/ is
tracked again (operator carries mock fixtures there).
- Partners list with name/domain/status/customers/margin + Create modal
- Partner detail: contract card, contact card, customers table, attach modal,
terminate (soft-delete) danger card
- Operator proxies for /partners + /partners/:slug/tenants
- platform-api: add partnerId Prop to Tenant schema. The field was being
silently dropped by Mongoose because the schema didn't declare it.
- tenants.service: rewrite update() to build $set/$unset explicitly and cast
partnerId via new Types.ObjectId(). Handles detach via $unset so the field
vanishes from the doc cleanly.
Operator can now manage tenants end-to-end from the UI:
- pages/tenants/index.vue — list with status/plan/domains/created/
provisioning-state columns, search by slug or name, status chips
with live counts (all/active/pending/suspended), click-through
to detail
- pages/tenants/[slug].vue — 7-tab detail (Overview, Users, Resources,
Billing, Audit, Support, Danger zone)
- 3 tabs hit real backends: Overview (identity + billing fields),
Users (lazy-loaded via new GET /tenants/:slug/users endpoint),
Resources (live provisioning state per integration + Reconcile button)
- 3 tabs render mock fixtures with warn-tone "mock" badges: Billing
(Stripe placeholder), Audit (sample log lines), Support (placeholder
pending the ticket queue work)
- Danger zone: 3 real-backend cards (Suspend / Resume / Soft-delete),
each gated by a ConfirmDialog modal. Verified live — clicked
Suspend on acme, status flipped to 'suspended' in Mongo, then
Resumed back to 'active'
platform-api additions:
- GET /tenants/:slug/users returns users with this tenant in their
tenantIds, sorted by last login. Same authorization rule as the
existing /tenants/:slug — platform admins always pass,
non-admins must be a member of the tenant
- tenants.module imports User schema for the new lookup
New components (apps/operator/components/):
- Tabs.vue — horizontal strip with optional per-tab counts, v-model
- ConfirmDialog.vue — Teleport-to-body modal, Escape/backdrop close,
danger/primary tone for the confirm button
Server proxy infrastructure (apps/operator/server/):
- utils/platform-api.ts — single helper encapsulating
access-token-from-session + bearer-forward + error normalization.
Every operator proxy route is now a one-liner against this helper
- api/tenants/index.get.ts, [slug]/{index.get,index.patch,index.delete,
users.get,suspend.post,resume.post,reconcile.post}.ts
Two real bugs found and fixed during the smoke test:
- Mongoose subdocument `_id` leaks into JSON when iterating
tenant.provisioningStatus. Switched to an explicit
`['authentik', 'stalwart', 'ocis']` whitelist in both v-fors
- Documents created before provisioningErrors was added (like the
acme tenant) don't have the field at all in JSON. Use optional
chaining (`tenant.provisioningErrors?.[k]`) instead of bracket
access. Without it: 'Cannot read properties of undefined (reading
"authentik")' during the Resources tab render
Operator portal now wears its real chrome instead of placeholder spans.
Sidebar + topbar + page header all rendered against the carbon palette
from tokens.css.
Components ported from the source design (operator-app.jsx,
platform-ui.jsx, operator-screens.jsx) as Vue 3 SFCs in
apps/operator/components/:
Foundation: NodeMark (copied from portal), UiIcon (expanded to 31 icons
covering sidebar/topbar/sort/arrows)
Primitives: Card (3 surface variants), UiButton (primary / secondary /
ghost / dark / danger × sm / md / lg), DataTable (header + rows),
Badge (7 tones), Avatar (deterministic palette by name hash), Mono,
Eyebrow, StatusDot, PageHeader (with actions slot)
Shell: OpSidebar (collapsible 232<->56px, 12 nav items in 4 sections,
active-row highlight from route, badge slot, brand + user footer);
OpTopbar (env badge with prod/staging/dev variants, palette trigger
stub for the ⌘K work in O.8, on-call pill, bell, avatar)
Layouts: layouts/default.vue wires sidebar + topbar + slot; layouts/blank.vue
is used by the login page (definePageMeta layout:'blank'). app.vue now
wraps NuxtPage in NuxtLayout (the missing piece — without it Nuxt warns
"Your project has layouts but the <NuxtLayout /> component has not been
used" and renders nothing chrome-wise).
Composable composables/useSidebar.ts holds the collapsed state shared
between OpSidebar's toggle button and layouts/default.vue's ⌘[ keyboard
shortcut.
Verified in the browser:
- Sidebar renders all 12 nav links with section dividers, env badge shows
PROD, PageHeader resolves to the user's display name from
useOidcAuth().user
- Collapse toggle flips sidebar width 232↔56; nav rows become icon-only
- Smoke test on the placeholder home still returns 409 for the seeded
test-partner (token forwarding survives the layout refactor)
Gotcha documented in the plan: Vite 7.3 added a strict
server.allowedHosts check that returns plaintext 403 for any host header
that isn't the dev origin. The customer portal pre-dates this Vite
version; operator needs allowedHosts: ['operator.dezky.local'] in
nuxt.config.ts under vite.server.
Pages/index.vue replaces the bare HTML placeholder from O.3 with the
new PageHeader + Card primitives — same smoke-test functionality, much
better visual fidelity.
Real screen content (Tenants, Partners, Infrastructure, etc.) lands in
O.5+. This commit is the chrome, the smoke test, and the verification
that the design system primitives compose correctly.
New Nuxt 3 app at apps/operator/ — internal admin portal on its own domain
(operator.dezky.local), own OAuth client (dezky-operator), own session
secrets, own cookies. Customer and operator surfaces can't decrypt each
other's session state.
OAuth flow verified end-to-end:
- GET / → middleware redirect to /auth/login
- User clicks Sign in → /auth/oidc/login → bounces to Authentik with
client_id=dezky-operator, scope includes 'groups'
- Authentik checks dezky-platform-admins group binding (added in O.1),
silent-reauths via the existing auth.dezky.local session
- Returns to /auth/oidc/callback with code, exchanges for token,
creates session cookie on operator.dezky.local
- Lands on pages/index.vue placeholder dashboard
Smoke test 'Create partner "test-partner"' button on the placeholder home
exercises the full operator-only authorization chain:
- 1st call: 200, partner created in Mongo
- 2nd call: 409 'already exists' (idempotency holds, token still valid)
- Same call from the customer portal: 403 'requires operator-scoped
token' (audience guard rejects dezky-portal aud)
JwtAuthGuard now multi-issuer in addition to multi-audience. Each
Authentik OAuth provider mints tokens with its own per-app iss URL
(.../application/o/<slug>/), so the guard accepts a comma-separated
AUTHENTIK_ISSUER. The audience-only fix from O.2 wasn't sufficient —
issuer is validated separately by jose.jwtVerify and was still pinned
to dezky-portal alone, yielding 'unexpected iss claim value' rejections.
Compose changes: new 'operator' service (Node 20 alpine, pnpm install +
nuxt dev, mkcert CA mount, traefik labels for operator.dezky.local +
TLS); new operator_node_modules volume; operator.dezky.local added to
traefik's Docker network aliases. Distinct OPERATOR_NUXT_OIDC_* session
secrets pulled from .env (gitignored, generated via openssl).
Real operator screens (sidebar, topbar, tenants, partners, etc.) come
in O.4. This commit is pure scaffolding + the security boundary proof.
JwtAuthGuard now accepts a comma-separated AUTHENTIK_AUDIENCE
('dezky-portal,dezky-operator'). jose.jwtVerify takes an array and succeeds
on any match — both customer-portal and operator-portal tokens validate
against this service. Per-endpoint guards restrict further.
New OperatorGuard enforces operator-only mutations:
1. JWT audience claim includes 'dezky-operator' (proof from the token
alone that this is a privileged session)
2. ActorService-resolved User has platformAdmin=true (DB check so
revocation works without waiting for the token to expire)
Both required; either alone is insufficient.
Partner module:
- Partner schema: slug, name, domain, status, marginPct, contactInfo,
billingInfo. marginPct is one number per partner (decided in grilling)
- CRUD endpoints under @UseGuards(JwtAuthGuard, OperatorGuard) — every
partner mutation requires operator scope
- GET /partners returns each row with a computed customers count from
aggregating Tenant.partnerId. MRR aggregation deferred until
Subscription gains a price column
- GET /partners/:slug/tenants for the partner detail view
- DELETE soft-terminates (status='terminated') — never hard-delete
because tenants may still reference the partner
Tenant changes:
- partnerId?: Types.ObjectId (ref Partner, indexed sparse) added to
Tenant schema
- UpdateTenantDto accepts partnerId so PATCH can attach/detach
- POST /tenants/:slug/suspend and /resume — operator-only via
OperatorGuard. PATCH already covers plan/domains/partnerId changes
Smoke test: customer-portal session sends POST /api/partners through the
portal proxy → 403 "This endpoint requires an operator-scoped token". The
positive test (operator-token → 200) waits for O.3 when there's an
operator app to mint the right token.
apps/portal/server/api/partners/index.post.ts is a temporary verification
proxy — delete once the operator portal exists.
What landed in Authentik (runtime state, not in git):
- OAuth2 provider 'dezky-operator', confidential, PKCE, audience
dezky-operator, redirect URIs operator.dezky.local/auth/oidc/{callback,logout}
- Application 'Dezky Operator' linked to the provider
- Policy binding: dezky-platform-admins group required on the application
.env (gitignored) gained OPERATOR_OIDC_CLIENT_ID/SECRET/ISSUER.
MFA-required is deferred — Authentik enforces it via a stage binding on
the auth flow, which is app-specific config better tackled when there's
a real enrollment to gate. akadmin already has WebAuthn so the flow
prompts for it anyway.
Discovery doc at /application/o/dezky-operator/.well-known/openid-
configuration confirmed: issuer correct, scopes include 'groups'.
Two gotchas documented in OPERATOR-PLAN.md:
- Authentik 2025.10 requires invalidation_flow alongside authorization_flow
- policies/group_membership endpoint is gone; use policies/bindings with a
direct group reference instead
O.0 prep from OPERATOR-PLAN.md. Mechanical refactor before adding partner
management and operator-specific endpoints. The service now owns more than
just provisioning orchestration (it'll soon own partners, tenant lifecycle
actions, multi-audience JWT validation), so the name 'platform-api' reflects
its scope better.
What changed:
- Directory: services/provisioning/ -> services/platform-api/
- Package: @dezky/provisioning -> @dezky/platform-api
- Docker: container_name dezky-provisioning -> dezky-platform-api;
compose service key 'provisioning' -> 'platform-api'; volume
provisioning_node_modules -> platform_api_node_modules
- Portal: PROVISIONING_INTERNAL_URL env var -> PLATFORM_API_INTERNAL_URL,
default URL http://provisioning:3001 -> http://platform-api:3001 in all
three proxy routes (me.get.ts, tenants/index.post.ts, tenants/[slug]/
reconcile.post.ts), plus NUXT_API_BASE updated
- Health endpoint service identifier and main.ts log lines updated to
'dezky-platform-api'
- Docs swept: README, CLAUDE.md, SERVICES.md, AUTHENTIK-SETUP.md,
NEXT-STEPS.md, TROUBLESHOOTING.md, OPERATOR-PLAN.md, traefik/dynamic.yml
What deliberately stays:
- Internal module names ProvisioningService / ProvisioningModule (those
describe an orchestration sub-concern, not the service's purpose)
- Tenant.provisioningStatus / provisioningErrors field names (state
per integration, not service name)
- File services/platform-api/src/tenants/provisioning.service.ts
- 'Hetzner provisioning' references in production-prep docs (infrastructure
provisioning, unrelated)
Verified end-to-end after rename: /api/me returns 200 with profile + 2
tenants + subscription, /api/tenants/dezky/reconcile returns 200 with
Authentik integration still ok.
OPERATOR-PLAN.md O.0 checkboxes ticked.
OPERATOR-PLAN.md records the decisions from the design review:
- Scope: C-visual (full UI fidelity, mock data for most screens) but real
CRUD for tenants and partners from day one
- Lives at apps/operator/ as a separate Nuxt app, separate domain, separate
Authentik OAuth client (dezky-operator), aud-claim distinguishes operator
vs portal tokens
- Backend stays as a single NestJS service; rename
services/provisioning -> services/platform-api as a prep commit
- Partner schema designed: slug/name/domain/status/marginPct/contactInfo;
Tenant gains optional partnerId; counts and MRR are computed at query time
- Impersonation: visual stub now (modal + banner, no-op toast); real OAuth
Token Exchange flow recorded as the first follow-up task
Also lists follow-up tasks (real audit log, feature flag backend, incident
management, partner portal) and out-of-scope items so the next grilling
session has a starting point.
Pointer added in NEXT-STEPS.md under a new 'Operator portal' track.
Honest status for the data-model + provisioning phases. Lists the smoke
test that verified the chain works end-to-end and points at the upstream
docs for the JMAP and libregraph follow-up work.
Implements Phase 3 from docs/NEXT-STEPS.md.
Mongoose schemas (services/provisioning/src/schemas/):
- Tenant: slug, name, status, plan, domains, billingInfo, plus handles for
Authentik group, OCIS space, and Stalwart domain (set in Phase 4)
- User: authentikSubjectId, tenantIds[], email, name, role, platformAdmin flag
- Subscription: tenantId, plan, status, Stripe IDs (unused until Phase 4)
Auth (services/provisioning/src/auth/):
- JwtAuthGuard verifies Authentik access tokens against the provider's JWKS
with issuer + audience checks. Uses NODE_EXTRA_CA_CERTS to trust the
mkcert root for the local Authentik cert
- ActorService resolves the verified JWT into a Mongo User document — every
controller reads tenantIds + platformAdmin from the DB, not the token
- CurrentUser decorator extracts the JWT payload onto controllers
CRUD modules:
- /tenants, /users, /subscriptions with create/read/update/delete
- /users/me upserts the caller's User record on every request, syncing email,
name, tenantIds, and platformAdmin from the JWT's groups claim — the only
place we read JWT.groups outside the bootstrap
Why DB-derived authz: putting all group memberships in the JWT doesn't scale
past ~50 tenants per user (header/cookie size limits, no mid-session
revocation, stale data until re-login). JWT now carries identity only; the
DB is the source of truth for who can see what.
Seed (SeedService.OnApplicationBootstrap): idempotent creation of the
default 'dezky' tenant + matching subscription. User records are created on
first /users/me hit.
Infrastructure:
- Traefik label exposes provisioning at https://api.dezky.local (dev only)
- api.dezky.local added to Docker network aliases on Traefik
- mkcert root CA mounted into the provisioning container for JWKS fetch
- Authentik 'groups' scope mapping created + attached to dezky-portal
provider; portal now requests it as a scope
- nuxt.config.ts portal: exposeAccessToken=true so Nitro forwards token;
NUXT_OIDC_TOKEN_KEY fixed to base64-encoded 32 bytes (was hex, causing
"Invalid key length" once exposeAccessToken turned on)
Portal: apps/portal/server/api/me.get.ts is a scaffolding route that
forwards the user's access token to provisioning and returns profile +
tenants + subscriptions — verifies the full chain end to end.
Brings up Dezky's local development environment end-to-end:
Infrastructure (docker-compose):
- Traefik v3.7 reverse proxy with mkcert TLS (v3.2 couldn't speak Docker API 1.54)
- Postgres + Mongo + Redis with healthchecks and init script for per-service users
- Authentik 2025.10 (server + worker) as OIDC IdP
- Stalwart v0.16 mail server (image renamed from stalwartlabs/mail-server)
- OCIS 7.0 with PROXY_TLS=false and OCIS_CONFIG_DIR=/etc/ocis so init writes
where the server reads
- Collabora office, plus the portal + provisioning service stubs
- Docker network aliases on Traefik so containers resolve auth.dezky.local etc.
through the network (not host /etc/hosts)
- Docker socket mount parameterized for macOS Docker Desktop symlink path
Authentik provisioning (done via API after stack boot):
- ocis-provider (public client) + OCIS Files application
- dezky-portal provider (confidential) + Dezky Portal application
- Admin API token bound to akadmin manually since 2025.10's
AUTHENTIK_BOOTSTRAP_TOKEN env var doesn't auto-materialize a token row
Portal (apps/portal):
- Nuxt 3 with nuxt-oidc-auth 1.0.0-beta.11 against generic 'oidc' preset
- Global auth middleware; login at /auth/oidc/login redirects to Authentik
- Visual implementation of Claude Design 'Auth' canvas: AuthShell, NodeMark,
Auth* sub-components, design tokens as CSS custom properties
- Pages: auth/login, auth/expired, auth/disabled, index (post-login landing)
- mkcert root CA mounted into the portal so Node fetch trusts Authentik's
self-signed cert (NODE_EXTRA_CA_CERTS) — dev only
Docs:
- AUTHENTIK-SETUP.md updated with manual token bind + portal provider scripted
alternative
- NEXT-STEPS.md: Phase 1 and Phase 2 marked done with file locations and
dev-mode caveats
Dev-mode shortcuts that need to be revisited before prod:
- skipAccessTokenParsing on the OIDC config
- NODE_EXTRA_CA_CERTS mkcert mount
- Bootstrap password still the generated value in .env
- Authentik admin token (dezky-bootstrap-token) is non-expiring
Ronni Baslund