feat(audit): cold-storage archival to S3 (Phase 4)
Final piece of the audit work. Events older than the hot retention window
move to S3-compatible object storage with signed manifests. Production uses
Hetzner Object Storage; dev uses a MinIO container with the same API.
Infra (infrastructure/docker-compose):
- New `minio` service exposing the S3 API at minio:9000 + admin console at
minio.dezky.local. Healthchecked. Bucket-init sidecar runs `mc mb` once
to create `dezky-audit`; safe to re-run.
- .env adds MINIO_ROOT_USER + MINIO_ROOT_PASSWORD.
- platform-api env: AUDIT_COLD_{ENDPOINT,REGION,BUCKET,ACCESS_KEY,SECRET_KEY}
+ AUDIT_HOT_RETENTION_DAYS=90 + ARCHIVE_ENABLED=false (dormant in dev;
operator UI's "Run archive now" bypasses this gate). AUDIT_COLD_SSE
opts into SSE-S3 — left unset in dev because MinIO without a KMS rejects
AES256 PUTs with "KMS is not configured".
Platform-api (services/platform-api/src/cold/):
- cold-storage.client.ts: thin @aws-sdk/client-s3 wrapper — put/head/list.
forcePathStyle=true so MinIO and Hetzner both work; same code, env-swap.
- archive.service.ts: runOnce() selects chained events with at < cutoff →
serializes to JSONL → gzip → sha256s → uploads JSONL + signed manifest
→ HEAD-confirms both objects exist → records an ArchiveBatch doc → only
then deletes from hot Mongo. Crash-safe: a failed upload leaves events
in hot. Manifest uses the Phase 3 AUDIT_SIGNING_KEY (HMAC-SHA-256), so
archives + checkpoints share trust chain. Bypassable via { override:
true } for the operator's UI force-run.
- archive.worker.ts: hourly tick guarded by configured run-hour-UTC
(default 03:00) + day-guard so the same UTC day doesn't archive twice.
Disabled until ARCHIVE_ENABLED=true.
- archive-batch.schema.ts: { archivedAt, startSeq, endSeq, eventCount,
manifestSha256, jsonlKey, manifestKey, bytesUncompressed }. The
manifest sha256 stored in Mongo lets us detect manifest tampering
without downloading the actual manifest.
Audit module additions:
- audit.controller.ts: GET /audit/archives, POST /audit/archive/run,
/audit/verify now reports { oldestHotSeq, highestArchivedSeq } so the
UI shows the tier boundary.
Operator UI (apps/operator):
- 2 new proxies: /api/audit/archives + /api/audit/archive/run (force
override=true). Both behind operator auth via the existing platformApi
helper.
- audit.vue: new "Cold storage" card with batch table (archived-at, seq
range, event count, size, truncated manifest sha256), "Run archive
now" button + per-run result line.
Smoke-tested end-to-end:
- 7 chained events in hot. /api/audit/archive/run → ok=true, batchId
returned. JSONL + manifest both exist in MinIO (verified via mc ls +
mc cat). Mongo's chained set went 7 → 0. Verify reports
highestArchivedSeq=1446 (since we burn-allocate seqs on Authentik
dup-key rejections). Operator /audit panel shows the batch with
manifest hash 1d8263…
- First attempt with SSE-S3 enabled failed cleanly (MinIO KMS not
configured) — archive service correctly left events in hot Mongo.
Made SSE opt-in via AUDIT_COLD_SSE=true; prod turns it on.
Out of scope (each could be its own session):
- Restore-to-hot endpoint (today: download from S3 + offline query)
- Client-side encryption (today: SSE-S3 in prod, none in dev)
- Multi-region replication
- Soft TTL safety net (defense-in-depth on top of app-managed deletion)
This completes the four-phase audit log work:
1. platform-api as audit hub
2. External system ingest (Authentik / Stalwart / OCIS)
3. Hash-chain + signed checkpoints (tamper evidence)
4. Cold-storage archival (retention without unbounded Mongo growth)
This commit is contained in:
@@ -72,13 +72,16 @@ function shortIp(ip?: string) {
|
||||
return ip.replace(/^::ffff:/, '')
|
||||
}
|
||||
|
||||
// ── Tamper-evidence (Phase 3) ──────────────────────────────────────────
|
||||
// ── Tamper-evidence (Phase 3) + cold-storage archives (Phase 4) ────────
|
||||
interface VerifyReport {
|
||||
ok: boolean
|
||||
totalEventsVerified: number
|
||||
checkpointsChecked: number
|
||||
latestCheckpointAt: string | null
|
||||
latestVerifiedSeq: number | null
|
||||
// Phase 4 additions — included in /audit/verify response.
|
||||
oldestHotSeq: number | null
|
||||
highestArchivedSeq: number | null
|
||||
break?:
|
||||
| { kind: 'event-hash-mismatch'; seq: number; expected: string; actual: string }
|
||||
| { kind: 'event-prev-hash-mismatch'; seq: number; expected: string; actual: string }
|
||||
@@ -90,13 +93,30 @@ interface CheckpointSummary {
|
||||
headHash: string | null
|
||||
reason?: string
|
||||
}
|
||||
interface ArchiveBatch {
|
||||
_id: string
|
||||
archivedAt: string
|
||||
startSeq: number
|
||||
endSeq: number
|
||||
eventCount: number
|
||||
manifestSha256: string
|
||||
jsonlKey: string
|
||||
manifestKey: string
|
||||
bytesUncompressed: number
|
||||
}
|
||||
|
||||
const { data: latestCp, refresh: refreshCp } = useLazyFetch<CheckpointSummary>(
|
||||
'/api/audit/checkpoint/latest',
|
||||
{ default: () => ({ at: null, headSeq: null, headHash: null }), server: false },
|
||||
)
|
||||
const { data: archives, refresh: refreshArchives } = useLazyFetch<ArchiveBatch[]>(
|
||||
'/api/audit/archives',
|
||||
{ default: () => [], server: false },
|
||||
)
|
||||
const verifyReport = ref<VerifyReport | null>(null)
|
||||
const verifying = ref(false)
|
||||
const archiving = ref(false)
|
||||
const archiveResult = ref<{ ok: boolean; reason?: string; eventCount?: number; startSeq?: number; endSeq?: number } | null>(null)
|
||||
|
||||
async function runVerify() {
|
||||
verifying.value = true
|
||||
@@ -113,6 +133,23 @@ async function forceCheckpoint() {
|
||||
await refreshCp()
|
||||
}
|
||||
|
||||
async function forceArchive() {
|
||||
archiving.value = true
|
||||
archiveResult.value = null
|
||||
try {
|
||||
archiveResult.value = await $fetch('/api/audit/archive/run', { method: 'POST' })
|
||||
await Promise.all([refreshArchives(), refresh()])
|
||||
} finally {
|
||||
archiving.value = false
|
||||
}
|
||||
}
|
||||
|
||||
function fmtBytes(n: number): string {
|
||||
if (n < 1024) return `${n} B`
|
||||
if (n < 1024 * 1024) return `${(n / 1024).toFixed(1)} KB`
|
||||
return `${(n / 1024 / 1024).toFixed(1)} MB`
|
||||
}
|
||||
|
||||
function fmtRelative(iso: string | null | undefined): string {
|
||||
if (!iso) return 'never'
|
||||
const ms = Date.now() - new Date(iso).getTime()
|
||||
@@ -259,10 +296,62 @@ function fmtRelative(iso: string | null | undefined): string {
|
||||
</div>
|
||||
</Card>
|
||||
|
||||
<!-- Cold-storage archives — Phase 4 -->
|
||||
<Card :pad="0" class="archive-card">
|
||||
<div class="archive-head">
|
||||
<div>
|
||||
<Eyebrow>Cold storage</Eyebrow>
|
||||
<div class="cap">
|
||||
{{
|
||||
archives?.length
|
||||
? `archived through seq ${archives[0].endSeq} · ${archives.length} batch${archives.length === 1 ? '' : 'es'}`
|
||||
: 'no archives yet · 90-day hot retention'
|
||||
}}
|
||||
</div>
|
||||
</div>
|
||||
<UiButton variant="secondary" :disabled="archiving" @click="forceArchive">
|
||||
{{ archiving ? 'Archiving…' : 'Run archive now' }}
|
||||
</UiButton>
|
||||
</div>
|
||||
|
||||
<div v-if="archiveResult" class="archive-result" :data-ok="archiveResult.ok">
|
||||
<Badge v-if="archiveResult.ok && archiveResult.eventCount" tone="ok" dot>archived</Badge>
|
||||
<Badge v-else-if="archiveResult.ok" tone="info" dot>no-op</Badge>
|
||||
<Badge v-else tone="bad" dot>failed</Badge>
|
||||
<Mono v-if="archiveResult.ok && archiveResult.eventCount">
|
||||
{{ archiveResult.eventCount }} event(s) · seq {{ archiveResult.startSeq }}–{{ archiveResult.endSeq }}
|
||||
</Mono>
|
||||
<Mono v-else dim>{{ archiveResult.reason || '—' }}</Mono>
|
||||
</div>
|
||||
|
||||
<table v-if="archives?.length">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Archived</th>
|
||||
<th>Seq range</th>
|
||||
<th>Events</th>
|
||||
<th>Size</th>
|
||||
<th>Manifest sha256</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr v-for="b in archives" :key="b._id">
|
||||
<td><Mono>{{ fmtAbs(b.archivedAt) }}</Mono></td>
|
||||
<td><Mono>{{ b.startSeq }}–{{ b.endSeq }}</Mono></td>
|
||||
<td><Mono>{{ b.eventCount }}</Mono></td>
|
||||
<td><Mono dim>{{ fmtBytes(b.bytesUncompressed) }}</Mono></td>
|
||||
<td><Mono dim>{{ b.manifestSha256.slice(0, 16) }}…</Mono></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<div v-else class="empty"><Mono dim>// no archive batches yet — events stay in hot Mongo for {{ '90' }} days, then move to S3 (MinIO in dev / Hetzner in prod)</Mono></div>
|
||||
</Card>
|
||||
|
||||
<Mono dim class="note">
|
||||
// sourced from /audit on platform-api · append-only · sha256 hash-chain
|
||||
with HMAC-signed checkpoints every 100 events or 5 minutes · retention
|
||||
+ cold-storage archival to Hetzner Object Storage is Phase 4
|
||||
// hot tier: Mongo · cold tier: S3-compatible object storage ·
|
||||
sha256 hash-chain with HMAC-signed checkpoints + signed archive
|
||||
manifests · retention 90 days hot, indefinite cold · production
|
||||
encryption at rest is SSE-S3
|
||||
</Mono>
|
||||
</div>
|
||||
</div>
|
||||
@@ -368,4 +457,26 @@ td.actor { display: flex; align-items: center; gap: 10px; }
|
||||
.verify-result[data-ok="true"] { background: rgba(31, 138, 91, 0.05); }
|
||||
.verify-result[data-ok="false"] { background: rgba(240, 88, 88, 0.06); }
|
||||
.result-line { display: flex; align-items: center; gap: 12px; flex-wrap: wrap; }
|
||||
|
||||
/* Cold-storage archives panel */
|
||||
.archive-card { margin-top: 8px; }
|
||||
.archive-head {
|
||||
padding: 14px 18px;
|
||||
display: flex; justify-content: space-between; align-items: center;
|
||||
gap: 16px;
|
||||
border-bottom: 1px solid var(--border);
|
||||
}
|
||||
.archive-head .cap { font-family: var(--font-display); font-weight: 600; font-size: 15px; margin-top: 2px; }
|
||||
.archive-result {
|
||||
padding: 10px 18px;
|
||||
display: flex; align-items: center; gap: 12px; flex-wrap: wrap;
|
||||
border-bottom: 1px solid var(--border);
|
||||
}
|
||||
.archive-result[data-ok="true"] { background: rgba(31, 138, 91, 0.05); }
|
||||
.archive-result[data-ok="false"] { background: rgba(240, 88, 88, 0.06); }
|
||||
.archive-card table { width: 100%; border-collapse: collapse; }
|
||||
.archive-card th, .archive-card td { padding: 10px 18px; font-size: 12px; text-align: left; }
|
||||
.archive-card th { font-family: var(--font-mono); font-size: 9px; letter-spacing: 0.12em; text-transform: uppercase; color: var(--text-mute); font-weight: 500; border-bottom: 1px solid var(--border); }
|
||||
.archive-card td { border-top: 1px solid var(--border); }
|
||||
.archive-card .empty { padding: 16px 18px; }
|
||||
</style>
|
||||
|
||||
@@ -0,0 +1,9 @@
|
||||
import { platformApi } from '~~/server/utils/platform-api'
|
||||
|
||||
export default defineEventHandler(async (event) => {
|
||||
// We force override=true here because this proxy is only callable from the
|
||||
// operator UI's "Run archive now" button, which is explicitly a dev/ops
|
||||
// exercise of the cold-storage path. Production may want to remove this
|
||||
// proxy entirely once schedulers are trusted.
|
||||
return platformApi(event, '/audit/archive/run?override=true', { method: 'POST' })
|
||||
})
|
||||
@@ -0,0 +1,3 @@
|
||||
import { platformApi } from '~~/server/utils/platform-api'
|
||||
|
||||
export default defineEventHandler((event) => platformApi(event, '/audit/archives'))
|
||||
@@ -32,6 +32,9 @@ volumes:
|
||||
portal_node_modules:
|
||||
platform_api_node_modules:
|
||||
operator_node_modules:
|
||||
# MinIO data (S3-compatible cold storage for audit archives). Production
|
||||
# swaps the endpoint to Hetzner Object Storage and this volume goes away.
|
||||
minio_data:
|
||||
|
||||
services:
|
||||
# ─────────────────────────────────────────────────────────────────
|
||||
@@ -127,6 +130,52 @@ services:
|
||||
timeout: 3s
|
||||
retries: 5
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────
|
||||
# MinIO — S3-compatible cold storage for audit archives (Phase 4).
|
||||
# Production swaps endpoint to Hetzner Object Storage; same protocol.
|
||||
# ─────────────────────────────────────────────────────────────────
|
||||
minio:
|
||||
image: minio/minio:latest
|
||||
container_name: dezky-minio
|
||||
restart: unless-stopped
|
||||
command: server /data --console-address ":9001"
|
||||
environment:
|
||||
MINIO_ROOT_USER: ${MINIO_ROOT_USER}
|
||||
MINIO_ROOT_PASSWORD: ${MINIO_ROOT_PASSWORD}
|
||||
volumes:
|
||||
- minio_data:/data
|
||||
networks: [dezky]
|
||||
healthcheck:
|
||||
test: ["CMD", "mc", "ready", "local"]
|
||||
interval: 10s
|
||||
timeout: 3s
|
||||
retries: 5
|
||||
labels:
|
||||
- traefik.enable=true
|
||||
# Optional: expose MinIO admin UI behind Traefik. Dev only — production
|
||||
# uses Hetzner's console.
|
||||
- traefik.http.routers.minio.rule=Host(`minio.dezky.local`)
|
||||
- traefik.http.routers.minio.tls=true
|
||||
- traefik.http.services.minio.loadbalancer.server.port=9001
|
||||
|
||||
# One-shot init container that creates the audit bucket if it doesn't
|
||||
# exist. Idempotent — re-running is a no-op. Exits cleanly so docker
|
||||
# doesn't restart it.
|
||||
minio-init:
|
||||
image: minio/mc:latest
|
||||
container_name: dezky-minio-init
|
||||
depends_on:
|
||||
minio:
|
||||
condition: service_healthy
|
||||
networks: [dezky]
|
||||
entrypoint: >
|
||||
sh -c "
|
||||
mc alias set local http://minio:9000 ${MINIO_ROOT_USER} ${MINIO_ROOT_PASSWORD} &&
|
||||
mc mb --ignore-existing local/dezky-audit &&
|
||||
echo 'MinIO bucket dezky-audit ready'
|
||||
"
|
||||
restart: "no"
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────
|
||||
# Authentik — Identity provider (OIDC/SAML SSO)
|
||||
# ─────────────────────────────────────────────────────────────────
|
||||
@@ -485,6 +534,18 @@ services:
|
||||
# out the current segment with a key-rotation checkpoint (not in scope
|
||||
# for Phase 3). Prod swaps HMAC for ed25519 from an HSM.
|
||||
AUDIT_SIGNING_KEY: ${AUDIT_SIGNING_KEY}
|
||||
# Cold storage (Phase 4). Dev uses MinIO on the docker network; prod
|
||||
# swaps endpoint to Hetzner Object Storage and provides real IAM keys.
|
||||
# ARCHIVE_ENABLED defaults to false in dev so the worker doesn't move
|
||||
# data we still want to query while building. The UI "Run archive now"
|
||||
# button bypasses this gate.
|
||||
AUDIT_COLD_ENDPOINT: http://minio:9000
|
||||
AUDIT_COLD_REGION: us-east-1
|
||||
AUDIT_COLD_BUCKET: dezky-audit
|
||||
AUDIT_COLD_ACCESS_KEY: ${MINIO_ROOT_USER}
|
||||
AUDIT_COLD_SECRET_KEY: ${MINIO_ROOT_PASSWORD}
|
||||
AUDIT_HOT_RETENTION_DAYS: "90"
|
||||
ARCHIVE_ENABLED: "false"
|
||||
volumes:
|
||||
- ../../services/platform-api:/app
|
||||
- platform_api_node_modules:/app/node_modules
|
||||
|
||||
@@ -13,6 +13,7 @@
|
||||
"typecheck": "tsc --noEmit"
|
||||
},
|
||||
"dependencies": {
|
||||
"@aws-sdk/client-s3": "^3.700.0",
|
||||
"@nestjs/common": "^10.4.0",
|
||||
"@nestjs/core": "^10.4.0",
|
||||
"@nestjs/platform-fastify": "^10.4.0",
|
||||
|
||||
Generated
+517
@@ -8,6 +8,9 @@ importers:
|
||||
|
||||
.:
|
||||
dependencies:
|
||||
'@aws-sdk/client-s3':
|
||||
specifier: ^3.700.0
|
||||
version: 3.1053.0
|
||||
'@nestjs/common':
|
||||
specifier: ^10.4.0
|
||||
version: 10.4.22(class-transformer@0.5.1)(class-validator@0.14.4)(reflect-metadata@0.2.2)(rxjs@7.8.2)
|
||||
@@ -78,6 +81,125 @@ packages:
|
||||
resolution: {integrity: sha512-I5wviiIqiFwar9Pdk30Lujk8FczEEc18i22A5c6Z9lbmhPQdTroDnEQdsfXjy404wPe8H62s0I15o4pmMGfTYQ==}
|
||||
engines: {node: ^18.13.0 || >=20.9.0, npm: ^6.11.0 || ^7.5.6 || >=8.0.0, yarn: '>= 1.13.0'}
|
||||
|
||||
'@aws-crypto/crc32@5.2.0':
|
||||
resolution: {integrity: sha512-nLbCWqQNgUiwwtFsen1AdzAtvuLRsQS8rYgMuxCrdKf9kOssamGLuPwyTY9wyYblNr9+1XM8v6zoDTPPSIeANg==}
|
||||
engines: {node: '>=16.0.0'}
|
||||
|
||||
'@aws-crypto/crc32c@5.2.0':
|
||||
resolution: {integrity: sha512-+iWb8qaHLYKrNvGRbiYRHSdKRWhto5XlZUEBwDjYNf+ly5SVYG6zEoYIdxvf5R3zyeP16w4PLBn3rH1xc74Rag==}
|
||||
|
||||
'@aws-crypto/sha1-browser@5.2.0':
|
||||
resolution: {integrity: sha512-OH6lveCFfcDjX4dbAvCFSYUjJZjDr/3XJ3xHtjn3Oj5b9RjojQo8npoLeA/bNwkOkrSQ0wgrHzXk4tDRxGKJeg==}
|
||||
|
||||
'@aws-crypto/sha256-browser@5.2.0':
|
||||
resolution: {integrity: sha512-AXfN/lGotSQwu6HNcEsIASo7kWXZ5HYWvfOmSNKDsEqC4OashTp8alTmaz+F7TC2L083SFv5RdB+qU3Vs1kZqw==}
|
||||
|
||||
'@aws-crypto/sha256-js@5.2.0':
|
||||
resolution: {integrity: sha512-FFQQyu7edu4ufvIZ+OadFpHHOt+eSTBaYaki44c+akjg7qZg9oOQeLlk77F6tSYqjDAFClrHJk9tMf0HdVyOvA==}
|
||||
engines: {node: '>=16.0.0'}
|
||||
|
||||
'@aws-crypto/supports-web-crypto@5.2.0':
|
||||
resolution: {integrity: sha512-iAvUotm021kM33eCdNfwIN//F77/IADDSs58i+MDaOqFrVjZo9bAal0NK7HurRuWLLpF1iLX7gbWrjHjeo+YFg==}
|
||||
|
||||
'@aws-crypto/util@5.2.0':
|
||||
resolution: {integrity: sha512-4RkU9EsI6ZpBve5fseQlGNUWKMa1RLPQ1dnjnQoe07ldfIzcsGb5hC5W0Dm7u423KWzawlrpbjXBrXCEv9zazQ==}
|
||||
|
||||
'@aws-sdk/client-s3@3.1053.0':
|
||||
resolution: {integrity: sha512-/oGxoB6p1Nqs935Blt+v1o+anSCEf2n3RjIrcLz84i4cn2Gr+Z7JpDdUkG5+74r5ctqEPG7k/phTGbJ9fNKnHg==}
|
||||
engines: {node: '>=20.0.0'}
|
||||
|
||||
'@aws-sdk/core@3.974.13':
|
||||
resolution: {integrity: sha512-+Y5/4tHki0uYgyx8eun146DegRVQBpdKGK5RbV0FTKJPpaKTchvqVxrrRFK6Wk0JksO4iAZKw3eqxGEIwtO98w==}
|
||||
engines: {node: '>=20.0.0'}
|
||||
|
||||
'@aws-sdk/crc64-nvme@3.972.9':
|
||||
resolution: {integrity: sha512-P+QGozmXn2mZZI7sDgk+aUm+RTI61MPSFB+Ir2vjEjEbEsE4e7hYtzrDvAUxZy9ko81h53e11+F/GYlvwDkaOQ==}
|
||||
engines: {node: '>=20.0.0'}
|
||||
|
||||
'@aws-sdk/credential-provider-env@3.972.39':
|
||||
resolution: {integrity: sha512-29wX9zpAvEt1vcj0psha+y6ygBHy2V/S72mp6e7q0KARLWXq+pwE/lR6qGkwknQvruh52lXvlqZIga8Hdxkucw==}
|
||||
engines: {node: '>=20.0.0'}
|
||||
|
||||
'@aws-sdk/credential-provider-http@3.972.41':
|
||||
resolution: {integrity: sha512-IA3CQTjtJkb6u1H4mE4936c8OPBMa9Jggtwe8U2Mqw/vvb/tZ5Ebd0mcZcX0uKWQhOyYo/+qNIwkV5Xh+FeJJA==}
|
||||
engines: {node: '>=20.0.0'}
|
||||
|
||||
'@aws-sdk/credential-provider-ini@3.972.43':
|
||||
resolution: {integrity: sha512-4mzII+3mZEVXXE1xzrLQrCJL7/r62A63bA6SVzZoNL5rqCJghpf+xgGltVrIBBs0n+mOZBKrQl2tRREtvZ5l6A==}
|
||||
engines: {node: '>=20.0.0'}
|
||||
|
||||
'@aws-sdk/credential-provider-login@3.972.43':
|
||||
resolution: {integrity: sha512-HG7kQCwXtbv3oBV61Ins0oNX8KKyvrMqqRkb6ZiAfQHbMuHaiNaEb2KnpKLPkNpqImSBK82UkVE/kaY6IfWikA==}
|
||||
engines: {node: '>=20.0.0'}
|
||||
|
||||
'@aws-sdk/credential-provider-node@3.972.44':
|
||||
resolution: {integrity: sha512-sDaBIT0yrNNIPfvlsiTCmANm07zKju+ipWODjEXgZlsjMeIJR3LVp7RDyAOzUoAsTbDfYKDWp+i5WrFiQP6rmQ==}
|
||||
engines: {node: '>=20.0.0'}
|
||||
|
||||
'@aws-sdk/credential-provider-process@3.972.39':
|
||||
resolution: {integrity: sha512-2k/amBifLd75eXNwgvPw/2lKYSQ3NhvHQgkVKVjfUq13/eJ3JRtHmznuFenn74OK3sSfp4SMy1YB2w+UVXoKqA==}
|
||||
engines: {node: '>=20.0.0'}
|
||||
|
||||
'@aws-sdk/credential-provider-sso@3.972.43':
|
||||
resolution: {integrity: sha512-LPc3+Y4vhH1T4x6CMqwCM6hk5+SRf/Lwmgm8INm95wxTtIRHcMwQUVkDzWu4Iw/RSncxYM2BC01OrYbxOPZvyg==}
|
||||
engines: {node: '>=20.0.0'}
|
||||
|
||||
'@aws-sdk/credential-provider-web-identity@3.972.43':
|
||||
resolution: {integrity: sha512-wQtL34lUD/09VXjwAUo2T+I3aEXRDxMB3DKmTJL/Zj0Gi6sLDTrVhae1XVt01yzkquOWajI/sZW72JGDZ1ciTw==}
|
||||
engines: {node: '>=20.0.0'}
|
||||
|
||||
'@aws-sdk/middleware-bucket-endpoint@3.972.15':
|
||||
resolution: {integrity: sha512-O2HDANa+MrvbxpaRVQDiH3T13uAa9AkMjKyZmDygwauAmmvqZ5B0iRmKW+fuVGW6NPXuyXurFgIx69lSvmAWGA==}
|
||||
engines: {node: '>=20.0.0'}
|
||||
|
||||
'@aws-sdk/middleware-expect-continue@3.972.13':
|
||||
resolution: {integrity: sha512-sHiqIFg8o2ipT7t40B89Vj0ubSUtY6OSt/+Ee/OXhHch5K4+81zP2+QX8Lkc/nJ2QSmCySxOke7TEbmX69fe2g==}
|
||||
engines: {node: '>=20.0.0'}
|
||||
|
||||
'@aws-sdk/middleware-flexible-checksums@3.974.21':
|
||||
resolution: {integrity: sha512-alAu9heyiBK/OmRNXVxq8mmPTgeW2AQ6EYjRsI38kPZa1MZvt2Jh+BlGq7/GG9OVXOaEgD7DlGj/Lzfy5OmuEg==}
|
||||
engines: {node: '>=20.0.0'}
|
||||
|
||||
'@aws-sdk/middleware-location-constraint@3.972.11':
|
||||
resolution: {integrity: sha512-hkfspNUP4criAH6ton6BGKgnm5dZx+7bUOy1YqlTfejDeUPAM23D81q/IX+hdlS3KUsfwGz5ADTqZWKBEUpf4A==}
|
||||
engines: {node: '>=20.0.0'}
|
||||
|
||||
'@aws-sdk/middleware-sdk-s3@3.972.42':
|
||||
resolution: {integrity: sha512-/xNqNGXv9LaxZd25L9VV4pnSOw9OdDNO4rAHamM+h3KQBSITljIH9vk3dveGga1I2j36lQd0rdG3gjNEXvtNew==}
|
||||
engines: {node: '>=20.0.0'}
|
||||
|
||||
'@aws-sdk/middleware-ssec@3.972.11':
|
||||
resolution: {integrity: sha512-7PQvGNhtveKlvVqNahqWx5yrwxP7ecwAoB1dYBf8eKwfo2tzzCbNnW+q2nO3N066ktQaB4iBQbDRWtizm+amoQ==}
|
||||
engines: {node: '>=20.0.0'}
|
||||
|
||||
'@aws-sdk/nested-clients@3.997.11':
|
||||
resolution: {integrity: sha512-nWXXJ1r/r8N2Gw1pWolRgED38/A9A8DHR2ETWIv220zh4PZHcybbR4hUVWWktmNXTRHzDJwRluapHn0rZxuoqA==}
|
||||
engines: {node: '>=20.0.0'}
|
||||
|
||||
'@aws-sdk/signature-v4-multi-region@3.996.28':
|
||||
resolution: {integrity: sha512-qs9z5LqXO/CZC2Lg9SGKpoLU8Rhi+m2pFKZqfO9pytX1clc0katqtsDNupJxFy0xT9wsZSPzM2v1y+/H/zfp5Q==}
|
||||
engines: {node: '>=20.0.0'}
|
||||
|
||||
'@aws-sdk/token-providers@3.1052.0':
|
||||
resolution: {integrity: sha512-QqZNB3so7UIDxZtroc85TQaLVxdZRFm0eWM1CSR2N+b06as9TOrilvrlTZuj3guYlxMs6yLOgGxnklJ5qMYtTw==}
|
||||
engines: {node: '>=20.0.0'}
|
||||
|
||||
'@aws-sdk/types@3.973.9':
|
||||
resolution: {integrity: sha512-kuBfgQVdcz5Bmapc4A13YbpVw/pXkesfhetcFYwbntqas8sF41OHyd4o28+/TG2ZQdHBsv90Lsu5y6oitvYCdg==}
|
||||
engines: {node: '>=20.0.0'}
|
||||
|
||||
'@aws-sdk/util-locate-window@3.965.5':
|
||||
resolution: {integrity: sha512-WhlJNNINQB+9qtLtZJcpQdgZw3SCDCpXdUJP7cToGwHbCWCnRckGlc6Bx/OhWwIYFNAn+FIydY8SZ0QmVu3xTQ==}
|
||||
engines: {node: '>=20.0.0'}
|
||||
|
||||
'@aws-sdk/xml-builder@3.972.25':
|
||||
resolution: {integrity: sha512-GH+Kjz4nPKWKHnsiQpnhP1MJdTGIcK4rAka6tzakgjjUkVgNsmPeEbbRAf09SzS1hjGu6duGHCBsxYke0BhHjQ==}
|
||||
engines: {node: '>=20.0.0'}
|
||||
|
||||
'@aws/lambda-invoke-store@0.2.4':
|
||||
resolution: {integrity: sha512-iY8yvjE0y651BixKNPgmv1WrQc+GZ142sb0z4gYnChDDY2YqI4P/jsSopBWrKfAt7LOJAkOXt7rC/hms+WclQQ==}
|
||||
engines: {node: '>=18.0.0'}
|
||||
|
||||
'@babel/code-frame@7.29.0':
|
||||
resolution: {integrity: sha512-9NhCeYjq9+3uxgdtp20LSiJXJvN0FeCtNGpJxuMFZ1Kv3cWUNb6DOhJwUvcVCzKGR66cw4njwM6hrJLqgOwbcw==}
|
||||
engines: {node: '>=6.9.0'}
|
||||
@@ -240,6 +362,9 @@ packages:
|
||||
'@nestjs/platform-express':
|
||||
optional: true
|
||||
|
||||
'@nodable/entities@2.1.0':
|
||||
resolution: {integrity: sha512-nyT7T3nbMyBI/lvr6L5TyWbFJAI9FTgVRakNoBqCD+PmID8DzFrrNdLLtHMwMszOtqZa8PAOV24ZqDnQrhQINA==}
|
||||
|
||||
'@nuxtjs/opencollective@0.3.2':
|
||||
resolution: {integrity: sha512-um0xL3fO7Mf4fDxcqx9KryrB7zgRM5JSlvGN5AGkP6JLM5XEKyjeAiPbNxdXVXQ16isuAhYpvP88NgL2BGd6aA==}
|
||||
engines: {node: '>=8.0.0', npm: '>=5.0.0'}
|
||||
@@ -252,6 +377,42 @@ packages:
|
||||
resolution: {integrity: sha512-+1VkjdD0QBLPodGrJUeqarH8VAIvQODIbwh9XpP5Syisf7YoQgsJKPNFoqqLQlu+VQ/tVSshMR6loPMn8U+dPg==}
|
||||
engines: {node: '>=14'}
|
||||
|
||||
'@smithy/core@3.24.4':
|
||||
resolution: {integrity: sha512-3UNRKEyQyAgVgM0LGlerCLm+ChZWZ1GPfde+jBEW6bm6bSBGU1p0EbblaUV3unbhwvidjLA5Zs3sOs7mnZwvAw==}
|
||||
engines: {node: '>=18.0.0'}
|
||||
|
||||
'@smithy/credential-provider-imds@4.3.4':
|
||||
resolution: {integrity: sha512-vKW0MEFRU4Y3MkVZUkpJm+g9qyPGLCXhc0YLggUdSdBB4g7IaSSsCE75P9rBXyWHrXY1UYSQUl8/DwsTR7QciA==}
|
||||
engines: {node: '>=18.0.0'}
|
||||
|
||||
'@smithy/fetch-http-handler@5.4.4':
|
||||
resolution: {integrity: sha512-qM7AUKI4G6d7lNgaZD3lA1tWSolh5r6gcixfTZAPstVURfjIbvreVTPz+994M0yC3HbX4YYhDRgr31Xy3XwWOQ==}
|
||||
engines: {node: '>=18.0.0'}
|
||||
|
||||
'@smithy/is-array-buffer@2.2.0':
|
||||
resolution: {integrity: sha512-GGP3O9QFD24uGeAXYUjwSTXARoqpZykHadOmA8G5vfJPK0/DC67qa//0qvqrJzL1xc8WQWX7/yc7fwudjPHPhA==}
|
||||
engines: {node: '>=14.0.0'}
|
||||
|
||||
'@smithy/node-http-handler@4.7.4':
|
||||
resolution: {integrity: sha512-HIeF+1vrDGzPkkv39Hj2vlHSXHY3p958jd/8ZnePIY6+ZOsQX8coyEUKO5yQu4r0bQIVsbpotVIrXXwyycMStQ==}
|
||||
engines: {node: '>=18.0.0'}
|
||||
|
||||
'@smithy/signature-v4@5.4.4':
|
||||
resolution: {integrity: sha512-e5UtkMvsatzBfbeBZjEOt0k0Z3BEsjTFL/n6fdO5vtBLe67tdy0dX7xw2DU7uZ3acwoHyeCqpU2Fzb7pxwHb6Q==}
|
||||
engines: {node: '>=18.0.0'}
|
||||
|
||||
'@smithy/types@4.14.2':
|
||||
resolution: {integrity: sha512-P+otAxbV4CqBybp7EkcJCrig63yE2E7PuNVOmilVMRcx/O+QDzGULTrKsq4DV13gSfak9ObPrWaHl/9bL5YcWw==}
|
||||
engines: {node: '>=18.0.0'}
|
||||
|
||||
'@smithy/util-buffer-from@2.2.0':
|
||||
resolution: {integrity: sha512-IJdWBbTcMQ6DA0gdNhh/BwrLkDR+ADW5Kr1aZmd4k3DIF6ezMV4R2NIAmT08wQJ3yUK82thHWmC/TnK/wpMMIA==}
|
||||
engines: {node: '>=14.0.0'}
|
||||
|
||||
'@smithy/util-utf8@2.3.0':
|
||||
resolution: {integrity: sha512-R8Rdn8Hy72KKcebgLiv8jQcQkXoLMOGGv5uI1/k0l+snqkOzQ1R0ChUBCxWMlBsFMekWjq0wRudIweFs7sKT5A==}
|
||||
engines: {node: '>=14.0.0'}
|
||||
|
||||
'@tokenizer/inflate@0.2.7':
|
||||
resolution: {integrity: sha512-MADQgmZT1eKjp06jpI2yozxaU9uVs4GzzgSL+uEq7bVcJ9V1ZXQkeGNql1fsSI0gMy1vhvNTNbUqrx+pZfJVmg==}
|
||||
engines: {node: '>=18'}
|
||||
@@ -455,6 +616,9 @@ packages:
|
||||
bl@4.1.0:
|
||||
resolution: {integrity: sha512-1W07cM9gS6DcLperZfFSj+bWLtaPGSOHWhPiGzXmvVJbRLdG82sH/Kn8EtW1VqWVA54AKf2h5k5BbnIbwF3h6w==}
|
||||
|
||||
bowser@2.14.1:
|
||||
resolution: {integrity: sha512-tzPjzCxygAKWFOJP011oxFHs57HzIhOEracIgAePE4pqB3LikALKnSzUyU4MGs9/iCEUuHlAJTjTc5M+u7YEGg==}
|
||||
|
||||
brace-expansion@1.1.14:
|
||||
resolution: {integrity: sha512-MWPGfDxnyzKU7rNOW9SP/c50vi3xrmrua/+6hfPbCS2ABNWfx24vPidzvC7krjU/RTo235sV776ymlsMtGKj8g==}
|
||||
|
||||
@@ -729,6 +893,13 @@ packages:
|
||||
fast-uri@3.1.2:
|
||||
resolution: {integrity: sha512-rVjf7ArG3LTk+FS6Yw81V1DLuZl1bRbNrev6Tmd/9RaroeeRRJhAt7jg/6YFxbvAQXUCavSoZhPPj6oOx+5KjQ==}
|
||||
|
||||
fast-xml-builder@1.2.0:
|
||||
resolution: {integrity: sha512-00aAWieqff+ZJhsXA4g1g7M8k+7AYoMUUHF+/zFb5U6Uv/P0Vl4QZo84/IcufzYalLuEj9928bXN9PbbFzMF0Q==}
|
||||
|
||||
fast-xml-parser@5.7.3:
|
||||
resolution: {integrity: sha512-C0AaNuC+mscy6vrAQKAc/rMq+zAPHodfHGZu4sGVehvAQt/JLG1O5zEcYcXSY5zSqr4YVgxsB+pHXTq0i7eDlg==}
|
||||
hasBin: true
|
||||
|
||||
fastify-plugin@4.5.1:
|
||||
resolution: {integrity: sha512-stRHYGeuqpEZTL1Ef0Ovr2ltazUT9g844X5z/zEBFLG8RYlpDiOCIG+ATvYEp+/zmc7sN29mcIMp8gvYplYPIQ==}
|
||||
|
||||
@@ -1131,6 +1302,10 @@ packages:
|
||||
resolution: {integrity: sha512-ayCKvm/phCGxOkYRSCM82iDwct8/EonSEgCSxWxD7ve6jHggsFl4fZVQBPRNgQoKiuV/odhFrGzQXZwbifC8Rg==}
|
||||
engines: {node: '>=8'}
|
||||
|
||||
path-expression-matcher@1.5.0:
|
||||
resolution: {integrity: sha512-cbrerZV+6rvdQrrD+iGMcZFEiiSrbv9Tfdkvnusy6y0x0GKBXREFg/Y65GhIfm0tnLntThhzCnfKwp1WRjeCyQ==}
|
||||
engines: {node: '>=14.0.0'}
|
||||
|
||||
path-key@3.1.1:
|
||||
resolution: {integrity: sha512-ojmeN0qd+y0jszEtoY48r0Peq5dwMEkIlCOu6Q5f41lfkswXuKtYrhgoTpLnyIcHm24Uhqx+5Tqm2InSwLhE6Q==}
|
||||
engines: {node: '>=8'}
|
||||
@@ -1348,6 +1523,9 @@ packages:
|
||||
resolution: {integrity: sha512-vavAMRXOgBVNF6nyEEmL3DBK19iRpDcoIwW+swQ+CbGiu7lju6t+JklA1MHweoWtadgt4ISVUsXLyDq34ddcwA==}
|
||||
engines: {node: '>=4'}
|
||||
|
||||
strnum@2.3.0:
|
||||
resolution: {integrity: sha512-ums3KNd42PGyx5xaoVTO1mjU1bH3NpY4vsrVlnv9PNGqQj8wd7rJ6nEypLrJ7z5vxK5RP0yMLo6J/Gsm62DI5Q==}
|
||||
|
||||
strtok3@10.3.5:
|
||||
resolution: {integrity: sha512-ki4hZQfh5rX0QDLLkOCj+h+CVNkqmp/CMf8v8kZpkNVK6jGQooMytqzLZYUVYIZcFZ6yDB70EfD8POcFXiF5oA==}
|
||||
engines: {node: '>=18'}
|
||||
@@ -1578,6 +1756,10 @@ packages:
|
||||
resolution: {integrity: sha512-si7QWI6zUMq56bESFvagtmzMdGOtoxfR+Sez11Mobfc7tm+VkUckk9bW2UeffTGVUbOksxmSw0AA2gs8g71NCQ==}
|
||||
engines: {node: '>=12'}
|
||||
|
||||
xml-naming@0.1.0:
|
||||
resolution: {integrity: sha512-k8KO9hrMyNk6tUWqUfkTEZbezRRpONVOzUTnc97VnCvyj6Tf9lyUR9EDAIeiVLv56jsMcoXEwjW8Kv5yPY52lw==}
|
||||
engines: {node: '>=16.0.0'}
|
||||
|
||||
yargs-parser@21.1.1:
|
||||
resolution: {integrity: sha512-tVpsJW7DdjecAiFpbIB1e3qxIQsE6NoPc5/eTdrbbIC4h0LVsWhnoa3g+m2HclBIujHzsxZ4VJVA+GUuc2/LBw==}
|
||||
engines: {node: '>=12'}
|
||||
@@ -1620,6 +1802,271 @@ snapshots:
|
||||
transitivePeerDependencies:
|
||||
- chokidar
|
||||
|
||||
'@aws-crypto/crc32@5.2.0':
|
||||
dependencies:
|
||||
'@aws-crypto/util': 5.2.0
|
||||
'@aws-sdk/types': 3.973.9
|
||||
tslib: 2.8.1
|
||||
|
||||
'@aws-crypto/crc32c@5.2.0':
|
||||
dependencies:
|
||||
'@aws-crypto/util': 5.2.0
|
||||
'@aws-sdk/types': 3.973.9
|
||||
tslib: 2.8.1
|
||||
|
||||
'@aws-crypto/sha1-browser@5.2.0':
|
||||
dependencies:
|
||||
'@aws-crypto/supports-web-crypto': 5.2.0
|
||||
'@aws-crypto/util': 5.2.0
|
||||
'@aws-sdk/types': 3.973.9
|
||||
'@aws-sdk/util-locate-window': 3.965.5
|
||||
'@smithy/util-utf8': 2.3.0
|
||||
tslib: 2.8.1
|
||||
|
||||
'@aws-crypto/sha256-browser@5.2.0':
|
||||
dependencies:
|
||||
'@aws-crypto/sha256-js': 5.2.0
|
||||
'@aws-crypto/supports-web-crypto': 5.2.0
|
||||
'@aws-crypto/util': 5.2.0
|
||||
'@aws-sdk/types': 3.973.9
|
||||
'@aws-sdk/util-locate-window': 3.965.5
|
||||
'@smithy/util-utf8': 2.3.0
|
||||
tslib: 2.8.1
|
||||
|
||||
'@aws-crypto/sha256-js@5.2.0':
|
||||
dependencies:
|
||||
'@aws-crypto/util': 5.2.0
|
||||
'@aws-sdk/types': 3.973.9
|
||||
tslib: 2.8.1
|
||||
|
||||
'@aws-crypto/supports-web-crypto@5.2.0':
|
||||
dependencies:
|
||||
tslib: 2.8.1
|
||||
|
||||
'@aws-crypto/util@5.2.0':
|
||||
dependencies:
|
||||
'@aws-sdk/types': 3.973.9
|
||||
'@smithy/util-utf8': 2.3.0
|
||||
tslib: 2.8.1
|
||||
|
||||
'@aws-sdk/client-s3@3.1053.0':
|
||||
dependencies:
|
||||
'@aws-crypto/sha1-browser': 5.2.0
|
||||
'@aws-crypto/sha256-browser': 5.2.0
|
||||
'@aws-crypto/sha256-js': 5.2.0
|
||||
'@aws-sdk/core': 3.974.13
|
||||
'@aws-sdk/credential-provider-node': 3.972.44
|
||||
'@aws-sdk/middleware-bucket-endpoint': 3.972.15
|
||||
'@aws-sdk/middleware-expect-continue': 3.972.13
|
||||
'@aws-sdk/middleware-flexible-checksums': 3.974.21
|
||||
'@aws-sdk/middleware-location-constraint': 3.972.11
|
||||
'@aws-sdk/middleware-sdk-s3': 3.972.42
|
||||
'@aws-sdk/middleware-ssec': 3.972.11
|
||||
'@aws-sdk/signature-v4-multi-region': 3.996.28
|
||||
'@aws-sdk/types': 3.973.9
|
||||
'@smithy/core': 3.24.4
|
||||
'@smithy/fetch-http-handler': 5.4.4
|
||||
'@smithy/node-http-handler': 4.7.4
|
||||
'@smithy/types': 4.14.2
|
||||
tslib: 2.8.1
|
||||
|
||||
'@aws-sdk/core@3.974.13':
|
||||
dependencies:
|
||||
'@aws-sdk/types': 3.973.9
|
||||
'@aws-sdk/xml-builder': 3.972.25
|
||||
'@aws/lambda-invoke-store': 0.2.4
|
||||
'@smithy/core': 3.24.4
|
||||
'@smithy/signature-v4': 5.4.4
|
||||
'@smithy/types': 4.14.2
|
||||
bowser: 2.14.1
|
||||
tslib: 2.8.1
|
||||
|
||||
'@aws-sdk/crc64-nvme@3.972.9':
|
||||
dependencies:
|
||||
'@smithy/types': 4.14.2
|
||||
tslib: 2.8.1
|
||||
|
||||
'@aws-sdk/credential-provider-env@3.972.39':
|
||||
dependencies:
|
||||
'@aws-sdk/core': 3.974.13
|
||||
'@aws-sdk/types': 3.973.9
|
||||
'@smithy/core': 3.24.4
|
||||
'@smithy/types': 4.14.2
|
||||
tslib: 2.8.1
|
||||
|
||||
'@aws-sdk/credential-provider-http@3.972.41':
|
||||
dependencies:
|
||||
'@aws-sdk/core': 3.974.13
|
||||
'@aws-sdk/types': 3.973.9
|
||||
'@smithy/core': 3.24.4
|
||||
'@smithy/fetch-http-handler': 5.4.4
|
||||
'@smithy/node-http-handler': 4.7.4
|
||||
'@smithy/types': 4.14.2
|
||||
tslib: 2.8.1
|
||||
|
||||
'@aws-sdk/credential-provider-ini@3.972.43':
|
||||
dependencies:
|
||||
'@aws-sdk/core': 3.974.13
|
||||
'@aws-sdk/credential-provider-env': 3.972.39
|
||||
'@aws-sdk/credential-provider-http': 3.972.41
|
||||
'@aws-sdk/credential-provider-login': 3.972.43
|
||||
'@aws-sdk/credential-provider-process': 3.972.39
|
||||
'@aws-sdk/credential-provider-sso': 3.972.43
|
||||
'@aws-sdk/credential-provider-web-identity': 3.972.43
|
||||
'@aws-sdk/nested-clients': 3.997.11
|
||||
'@aws-sdk/types': 3.973.9
|
||||
'@smithy/core': 3.24.4
|
||||
'@smithy/credential-provider-imds': 4.3.4
|
||||
'@smithy/types': 4.14.2
|
||||
tslib: 2.8.1
|
||||
|
||||
'@aws-sdk/credential-provider-login@3.972.43':
|
||||
dependencies:
|
||||
'@aws-sdk/core': 3.974.13
|
||||
'@aws-sdk/nested-clients': 3.997.11
|
||||
'@aws-sdk/types': 3.973.9
|
||||
'@smithy/core': 3.24.4
|
||||
'@smithy/types': 4.14.2
|
||||
tslib: 2.8.1
|
||||
|
||||
'@aws-sdk/credential-provider-node@3.972.44':
|
||||
dependencies:
|
||||
'@aws-sdk/credential-provider-env': 3.972.39
|
||||
'@aws-sdk/credential-provider-http': 3.972.41
|
||||
'@aws-sdk/credential-provider-ini': 3.972.43
|
||||
'@aws-sdk/credential-provider-process': 3.972.39
|
||||
'@aws-sdk/credential-provider-sso': 3.972.43
|
||||
'@aws-sdk/credential-provider-web-identity': 3.972.43
|
||||
'@aws-sdk/types': 3.973.9
|
||||
'@smithy/core': 3.24.4
|
||||
'@smithy/credential-provider-imds': 4.3.4
|
||||
'@smithy/types': 4.14.2
|
||||
tslib: 2.8.1
|
||||
|
||||
'@aws-sdk/credential-provider-process@3.972.39':
|
||||
dependencies:
|
||||
'@aws-sdk/core': 3.974.13
|
||||
'@aws-sdk/types': 3.973.9
|
||||
'@smithy/core': 3.24.4
|
||||
'@smithy/types': 4.14.2
|
||||
tslib: 2.8.1
|
||||
|
||||
'@aws-sdk/credential-provider-sso@3.972.43':
|
||||
dependencies:
|
||||
'@aws-sdk/core': 3.974.13
|
||||
'@aws-sdk/nested-clients': 3.997.11
|
||||
'@aws-sdk/token-providers': 3.1052.0
|
||||
'@aws-sdk/types': 3.973.9
|
||||
'@smithy/core': 3.24.4
|
||||
'@smithy/types': 4.14.2
|
||||
tslib: 2.8.1
|
||||
|
||||
'@aws-sdk/credential-provider-web-identity@3.972.43':
|
||||
dependencies:
|
||||
'@aws-sdk/core': 3.974.13
|
||||
'@aws-sdk/nested-clients': 3.997.11
|
||||
'@aws-sdk/types': 3.973.9
|
||||
'@smithy/core': 3.24.4
|
||||
'@smithy/types': 4.14.2
|
||||
tslib: 2.8.1
|
||||
|
||||
'@aws-sdk/middleware-bucket-endpoint@3.972.15':
|
||||
dependencies:
|
||||
'@aws-sdk/core': 3.974.13
|
||||
'@aws-sdk/types': 3.973.9
|
||||
'@smithy/core': 3.24.4
|
||||
'@smithy/types': 4.14.2
|
||||
tslib: 2.8.1
|
||||
|
||||
'@aws-sdk/middleware-expect-continue@3.972.13':
|
||||
dependencies:
|
||||
'@aws-sdk/types': 3.973.9
|
||||
'@smithy/core': 3.24.4
|
||||
'@smithy/types': 4.14.2
|
||||
tslib: 2.8.1
|
||||
|
||||
'@aws-sdk/middleware-flexible-checksums@3.974.21':
|
||||
dependencies:
|
||||
'@aws-crypto/crc32': 5.2.0
|
||||
'@aws-crypto/crc32c': 5.2.0
|
||||
'@aws-crypto/util': 5.2.0
|
||||
'@aws-sdk/core': 3.974.13
|
||||
'@aws-sdk/crc64-nvme': 3.972.9
|
||||
'@aws-sdk/types': 3.973.9
|
||||
'@smithy/core': 3.24.4
|
||||
'@smithy/types': 4.14.2
|
||||
tslib: 2.8.1
|
||||
|
||||
'@aws-sdk/middleware-location-constraint@3.972.11':
|
||||
dependencies:
|
||||
'@aws-sdk/types': 3.973.9
|
||||
'@smithy/types': 4.14.2
|
||||
tslib: 2.8.1
|
||||
|
||||
'@aws-sdk/middleware-sdk-s3@3.972.42':
|
||||
dependencies:
|
||||
'@aws-sdk/core': 3.974.13
|
||||
'@aws-sdk/signature-v4-multi-region': 3.996.28
|
||||
'@aws-sdk/types': 3.973.9
|
||||
'@smithy/core': 3.24.4
|
||||
'@smithy/signature-v4': 5.4.4
|
||||
'@smithy/types': 4.14.2
|
||||
tslib: 2.8.1
|
||||
|
||||
'@aws-sdk/middleware-ssec@3.972.11':
|
||||
dependencies:
|
||||
'@aws-sdk/types': 3.973.9
|
||||
'@smithy/types': 4.14.2
|
||||
tslib: 2.8.1
|
||||
|
||||
'@aws-sdk/nested-clients@3.997.11':
|
||||
dependencies:
|
||||
'@aws-crypto/sha256-browser': 5.2.0
|
||||
'@aws-crypto/sha256-js': 5.2.0
|
||||
'@aws-sdk/core': 3.974.13
|
||||
'@aws-sdk/signature-v4-multi-region': 3.996.28
|
||||
'@aws-sdk/types': 3.973.9
|
||||
'@smithy/core': 3.24.4
|
||||
'@smithy/fetch-http-handler': 5.4.4
|
||||
'@smithy/node-http-handler': 4.7.4
|
||||
'@smithy/types': 4.14.2
|
||||
tslib: 2.8.1
|
||||
|
||||
'@aws-sdk/signature-v4-multi-region@3.996.28':
|
||||
dependencies:
|
||||
'@aws-sdk/types': 3.973.9
|
||||
'@smithy/core': 3.24.4
|
||||
'@smithy/signature-v4': 5.4.4
|
||||
'@smithy/types': 4.14.2
|
||||
tslib: 2.8.1
|
||||
|
||||
'@aws-sdk/token-providers@3.1052.0':
|
||||
dependencies:
|
||||
'@aws-sdk/core': 3.974.13
|
||||
'@aws-sdk/nested-clients': 3.997.11
|
||||
'@aws-sdk/types': 3.973.9
|
||||
'@smithy/core': 3.24.4
|
||||
'@smithy/types': 4.14.2
|
||||
tslib: 2.8.1
|
||||
|
||||
'@aws-sdk/types@3.973.9':
|
||||
dependencies:
|
||||
'@smithy/types': 4.14.2
|
||||
tslib: 2.8.1
|
||||
|
||||
'@aws-sdk/util-locate-window@3.965.5':
|
||||
dependencies:
|
||||
tslib: 2.8.1
|
||||
|
||||
'@aws-sdk/xml-builder@3.972.25':
|
||||
dependencies:
|
||||
'@nodable/entities': 2.1.0
|
||||
'@smithy/types': 4.14.2
|
||||
fast-xml-parser: 5.7.3
|
||||
tslib: 2.8.1
|
||||
|
||||
'@aws/lambda-invoke-store@0.2.4': {}
|
||||
|
||||
'@babel/code-frame@7.29.0':
|
||||
dependencies:
|
||||
'@babel/helper-validator-identifier': 7.28.5
|
||||
@@ -1820,6 +2267,8 @@ snapshots:
|
||||
'@nestjs/core': 10.4.22(@nestjs/common@10.4.22(class-transformer@0.5.1)(class-validator@0.14.4)(reflect-metadata@0.2.2)(rxjs@7.8.2))(reflect-metadata@0.2.2)(rxjs@7.8.2)
|
||||
tslib: 2.8.1
|
||||
|
||||
'@nodable/entities@2.1.0': {}
|
||||
|
||||
'@nuxtjs/opencollective@0.3.2':
|
||||
dependencies:
|
||||
chalk: 4.1.2
|
||||
@@ -1833,6 +2282,54 @@ snapshots:
|
||||
'@pkgjs/parseargs@0.11.0':
|
||||
optional: true
|
||||
|
||||
'@smithy/core@3.24.4':
|
||||
dependencies:
|
||||
'@aws-crypto/crc32': 5.2.0
|
||||
'@smithy/types': 4.14.2
|
||||
tslib: 2.8.1
|
||||
|
||||
'@smithy/credential-provider-imds@4.3.4':
|
||||
dependencies:
|
||||
'@smithy/core': 3.24.4
|
||||
'@smithy/types': 4.14.2
|
||||
tslib: 2.8.1
|
||||
|
||||
'@smithy/fetch-http-handler@5.4.4':
|
||||
dependencies:
|
||||
'@smithy/core': 3.24.4
|
||||
'@smithy/types': 4.14.2
|
||||
tslib: 2.8.1
|
||||
|
||||
'@smithy/is-array-buffer@2.2.0':
|
||||
dependencies:
|
||||
tslib: 2.8.1
|
||||
|
||||
'@smithy/node-http-handler@4.7.4':
|
||||
dependencies:
|
||||
'@smithy/core': 3.24.4
|
||||
'@smithy/types': 4.14.2
|
||||
tslib: 2.8.1
|
||||
|
||||
'@smithy/signature-v4@5.4.4':
|
||||
dependencies:
|
||||
'@smithy/core': 3.24.4
|
||||
'@smithy/types': 4.14.2
|
||||
tslib: 2.8.1
|
||||
|
||||
'@smithy/types@4.14.2':
|
||||
dependencies:
|
||||
tslib: 2.8.1
|
||||
|
||||
'@smithy/util-buffer-from@2.2.0':
|
||||
dependencies:
|
||||
'@smithy/is-array-buffer': 2.2.0
|
||||
tslib: 2.8.1
|
||||
|
||||
'@smithy/util-utf8@2.3.0':
|
||||
dependencies:
|
||||
'@smithy/util-buffer-from': 2.2.0
|
||||
tslib: 2.8.1
|
||||
|
||||
'@tokenizer/inflate@0.2.7':
|
||||
dependencies:
|
||||
debug: 4.4.3
|
||||
@@ -2055,6 +2552,8 @@ snapshots:
|
||||
inherits: 2.0.4
|
||||
readable-stream: 3.6.2
|
||||
|
||||
bowser@2.14.1: {}
|
||||
|
||||
brace-expansion@1.1.14:
|
||||
dependencies:
|
||||
balanced-match: 1.0.2
|
||||
@@ -2305,6 +2804,18 @@ snapshots:
|
||||
|
||||
fast-uri@3.1.2: {}
|
||||
|
||||
fast-xml-builder@1.2.0:
|
||||
dependencies:
|
||||
path-expression-matcher: 1.5.0
|
||||
xml-naming: 0.1.0
|
||||
|
||||
fast-xml-parser@5.7.3:
|
||||
dependencies:
|
||||
'@nodable/entities': 2.1.0
|
||||
fast-xml-builder: 1.2.0
|
||||
path-expression-matcher: 1.5.0
|
||||
strnum: 2.3.0
|
||||
|
||||
fastify-plugin@4.5.1: {}
|
||||
|
||||
fastify@4.28.1:
|
||||
@@ -2726,6 +3237,8 @@ snapshots:
|
||||
json-parse-even-better-errors: 2.3.1
|
||||
lines-and-columns: 1.2.4
|
||||
|
||||
path-expression-matcher@1.5.0: {}
|
||||
|
||||
path-key@3.1.1: {}
|
||||
|
||||
path-scurry@1.11.1:
|
||||
@@ -2920,6 +3433,8 @@ snapshots:
|
||||
|
||||
strip-bom@3.0.0: {}
|
||||
|
||||
strnum@2.3.0: {}
|
||||
|
||||
strtok3@10.3.5:
|
||||
dependencies:
|
||||
'@tokenizer/token': 0.3.0
|
||||
@@ -3134,6 +3649,8 @@ snapshots:
|
||||
string-width: 5.1.2
|
||||
strip-ansi: 7.2.0
|
||||
|
||||
xml-naming@0.1.0: {}
|
||||
|
||||
yargs-parser@21.1.1: {}
|
||||
|
||||
yn@3.1.1: {}
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { Controller, Get, Post, Query, UseGuards } from '@nestjs/common'
|
||||
import { JwtAuthGuard } from '../auth/jwt-auth.guard.js'
|
||||
import { OperatorGuard } from '../auth/operator.guard.js'
|
||||
import { ArchiveService } from '../cold/archive.service.js'
|
||||
import { ListAuditDto } from './dto/list-audit.dto.js'
|
||||
import { AuditService } from './audit.service.js'
|
||||
import { CheckpointService } from './checkpoint.service.js'
|
||||
@@ -11,10 +12,15 @@ import { AuditVerifier } from './verifier.service.js'
|
||||
// mutation in other modules. Operator-only because the trail is sensitive.
|
||||
//
|
||||
// Phase 3 surfaces:
|
||||
// GET /audit/verify — walks the chain + validates checkpoint signatures
|
||||
// GET /audit/checkpoint/latest — current "last verified" anchor for the UI
|
||||
// POST /audit/checkpoint — force a fresh checkpoint (rare; testing
|
||||
// the chain or anchoring before an export)
|
||||
// GET /audit/verify — walks chain + validates checkpoint sigs
|
||||
// GET /audit/checkpoint/latest — current "last verified" anchor for UI
|
||||
// POST /audit/checkpoint — force a fresh checkpoint
|
||||
// Phase 4 surfaces:
|
||||
// GET /audit/archives — list archive batches (most recent first)
|
||||
// POST /audit/archive/run — manually trigger an archive run with
|
||||
// override=true so the retention window
|
||||
// is bypassed (used from the operator UI
|
||||
// to exercise the cold-storage path in dev)
|
||||
@Controller('audit')
|
||||
@UseGuards(JwtAuthGuard, OperatorGuard)
|
||||
export class AuditController {
|
||||
@@ -22,6 +28,7 @@ export class AuditController {
|
||||
private readonly audit: AuditService,
|
||||
private readonly checkpoints: CheckpointService,
|
||||
private readonly verifier: AuditVerifier,
|
||||
private readonly archives: ArchiveService,
|
||||
) {}
|
||||
|
||||
@Get()
|
||||
@@ -41,8 +48,15 @@ export class AuditController {
|
||||
}
|
||||
|
||||
@Get('verify')
|
||||
verify() {
|
||||
return this.verifier.verify()
|
||||
async verify() {
|
||||
const base = await this.verifier.verify()
|
||||
// Augment with tier boundary so the UI can show "events before X are
|
||||
// archived, events from X+ are queryable here".
|
||||
const [oldestHotSeq, highestArchivedSeq] = await Promise.all([
|
||||
this.archives.oldestHotSeq(),
|
||||
this.archives.highestArchivedSeq(),
|
||||
])
|
||||
return { ...base, oldestHotSeq, highestArchivedSeq }
|
||||
}
|
||||
|
||||
@Get('checkpoint/latest')
|
||||
@@ -56,4 +70,30 @@ export class AuditController {
|
||||
forceCheckpoint() {
|
||||
return this.checkpoints.tryWrite('manual')
|
||||
}
|
||||
|
||||
@Get('archives')
|
||||
async listArchives() {
|
||||
const batches = await this.archives.listBatches(100)
|
||||
return batches.map((b) => ({
|
||||
_id: b._id,
|
||||
archivedAt: b.archivedAt,
|
||||
startSeq: b.startSeq,
|
||||
endSeq: b.endSeq,
|
||||
eventCount: b.eventCount,
|
||||
manifestSha256: b.manifestSha256,
|
||||
jsonlKey: b.jsonlKey,
|
||||
manifestKey: b.manifestKey,
|
||||
bytesUncompressed: b.bytesUncompressed,
|
||||
}))
|
||||
}
|
||||
|
||||
// `override=true` (the only mode we expose to the UI today) bypasses the
|
||||
// 90-day retention check and archives everything older than now. This is
|
||||
// how operators exercise the cold-storage path in dev without 3-month-old
|
||||
// events. Production should disable this query param via a config flag
|
||||
// before going live; for now it's gated behind OperatorGuard.
|
||||
@Post('archive/run')
|
||||
async runArchive(@Query('override') override?: string) {
|
||||
return this.archives.runOnce({ override: override === 'true' })
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { Module } from '@nestjs/common'
|
||||
import { MongooseModule } from '@nestjs/mongoose'
|
||||
import { AuthModule } from '../auth/auth.module.js'
|
||||
import { ColdModule } from '../cold/cold.module.js'
|
||||
import {
|
||||
AuditCheckpoint,
|
||||
AuditCheckpointSchema,
|
||||
@@ -18,6 +19,7 @@ import { AuditVerifier } from './verifier.service.js'
|
||||
@Module({
|
||||
imports: [
|
||||
AuthModule,
|
||||
ColdModule,
|
||||
MongooseModule.forFeature([
|
||||
{ name: AuditEvent.name, schema: AuditEventSchema },
|
||||
{ name: AuditCounter.name, schema: AuditCounterSchema },
|
||||
|
||||
@@ -0,0 +1,261 @@
|
||||
// Moves chained audit events older than AUDIT_HOT_RETENTION_DAYS from Mongo
|
||||
// to cold storage. Idempotent and crash-safe — events are deleted ONLY after
|
||||
// the JSONL + manifest have been confirmed in S3 (HEAD check). A failed
|
||||
// upload or HEAD check leaves the events in hot Mongo for the next run.
|
||||
//
|
||||
// Trust chain across tiers:
|
||||
// - Hot events carry sha256 hash + prev-hash links (Phase 3).
|
||||
// - When archived, the JSONL preserves these fields verbatim.
|
||||
// - The manifest records the JSONL's sha256 + the boundary event hashes,
|
||||
// signed with AUDIT_SIGNING_KEY (same key as Phase 3 checkpoints).
|
||||
// - The Phase 3 checkpoint that covers the archived seq range still
|
||||
// verifies — we're not orphaning chain integrity, just moving the
|
||||
// events themselves.
|
||||
|
||||
import { Injectable, Logger } from '@nestjs/common'
|
||||
import { ConfigService } from '@nestjs/config'
|
||||
import { InjectModel } from '@nestjs/mongoose'
|
||||
import { createHash, createHmac } from 'node:crypto'
|
||||
import { gzipSync } from 'node:zlib'
|
||||
import type { Model } from 'mongoose'
|
||||
import {
|
||||
AuditEvent,
|
||||
type AuditEventDocument,
|
||||
} from '../schemas/audit-event.schema.js'
|
||||
import {
|
||||
ArchiveBatch,
|
||||
type ArchiveBatchDocument,
|
||||
} from '../schemas/archive-batch.schema.js'
|
||||
import { ColdStorageClient } from './cold-storage.client.js'
|
||||
|
||||
export interface ArchiveResult {
|
||||
ok: boolean
|
||||
reason?: string
|
||||
batchId?: string
|
||||
startSeq?: number
|
||||
endSeq?: number
|
||||
eventCount?: number
|
||||
}
|
||||
|
||||
interface ArchiveManifest {
|
||||
version: 1
|
||||
startSeq: number
|
||||
endSeq: number
|
||||
eventCount: number
|
||||
startedAt: string
|
||||
endedAt: string
|
||||
bytesUncompressed: number
|
||||
bytesCompressed: number
|
||||
sha256: string // sha256 of the JSONL (uncompressed)
|
||||
firstEventHash: string
|
||||
lastEventHash: string
|
||||
signature: string
|
||||
sigAlg: 'HMAC-SHA-256'
|
||||
}
|
||||
|
||||
@Injectable()
|
||||
export class ArchiveService {
|
||||
private readonly logger = new Logger(ArchiveService.name)
|
||||
private readonly retentionDays: number
|
||||
private readonly signingKey: string
|
||||
// Server-side encryption — opt-in via AUDIT_COLD_SSE=true. Production
|
||||
// (Hetzner Object Storage) supports SSE-S3 natively. Dev/MinIO without
|
||||
// a KMS rejects it with "KMS is not configured" so the default is off.
|
||||
// Client-side encryption is a larger workstream tracked for later.
|
||||
private readonly serverSideEncryption: 'AES256' | undefined
|
||||
|
||||
constructor(
|
||||
@InjectModel(AuditEvent.name) private readonly events: Model<AuditEventDocument>,
|
||||
@InjectModel(ArchiveBatch.name) private readonly batches: Model<ArchiveBatchDocument>,
|
||||
private readonly cold: ColdStorageClient,
|
||||
config: ConfigService,
|
||||
) {
|
||||
this.retentionDays = Number(config.get('AUDIT_HOT_RETENTION_DAYS') ?? 90)
|
||||
this.signingKey = config.getOrThrow<string>('AUDIT_SIGNING_KEY')
|
||||
this.serverSideEncryption = config.get('AUDIT_COLD_SSE') === 'true' ? 'AES256' : undefined
|
||||
}
|
||||
|
||||
// One archive run. Selects events older than the retention boundary, packs
|
||||
// them, uploads, then deletes from hot Mongo. Returns a small result so
|
||||
// the worker can log a meaningful line.
|
||||
async runOnce(opts?: { override?: boolean }): Promise<ArchiveResult> {
|
||||
// Cutoff: events with at < cutoff go cold. For dev/forced runs, the
|
||||
// caller can override the retention check entirely (so we can exercise
|
||||
// the path on minute-old events).
|
||||
const cutoff = opts?.override
|
||||
? new Date()
|
||||
: new Date(Date.now() - this.retentionDays * 24 * 60 * 60 * 1000)
|
||||
|
||||
// Pull the eligible events ordered by seq. Cap per run so a runaway
|
||||
// backlog doesn't OOM us — at 100k events/batch the next run picks up
|
||||
// where we left off.
|
||||
const MAX_PER_BATCH = 100_000
|
||||
const events = await this.events
|
||||
.find({ chained: true, at: { $lt: cutoff } })
|
||||
.sort({ seq: 1 })
|
||||
.limit(MAX_PER_BATCH)
|
||||
.lean()
|
||||
.exec()
|
||||
|
||||
if (events.length === 0) {
|
||||
return { ok: true, reason: 'no events older than cutoff' }
|
||||
}
|
||||
|
||||
const startSeq = events[0].seq!
|
||||
const endSeq = events[events.length - 1].seq!
|
||||
const startedAt = new Date()
|
||||
|
||||
// Build JSONL — one event per line, no extra wrapping. Preserves the
|
||||
// original document shape so offline tools can `jq` it directly.
|
||||
const jsonlLines = events.map((e) => JSON.stringify(stripMongoMeta(e)))
|
||||
const jsonl = jsonlLines.join('\n') + '\n'
|
||||
const bytesUncompressed = Buffer.byteLength(jsonl, 'utf8')
|
||||
const gzipped = gzipSync(Buffer.from(jsonl, 'utf8'))
|
||||
|
||||
// Hash the uncompressed payload — the manifest's hash is content-
|
||||
// addressable so anyone redownloading + decompressing can re-verify.
|
||||
const sha256 = createHash('sha256').update(jsonl).digest('hex')
|
||||
|
||||
const objectPrefix = objectKeyPrefix(startedAt, startSeq, endSeq)
|
||||
const jsonlKey = `${objectPrefix}.jsonl.gz`
|
||||
const manifestKey = `${objectPrefix}.manifest.json`
|
||||
|
||||
const firstEventHash = events[0].hash ?? '<missing>'
|
||||
const lastEventHash = events[events.length - 1].hash ?? '<missing>'
|
||||
const endedAt = new Date()
|
||||
|
||||
// Sign the manifest. Same key as Phase 3 checkpoints — an attacker who
|
||||
// wants to forge an archive batch needs both S3 write AND this key.
|
||||
const unsigned = {
|
||||
version: 1 as const,
|
||||
startSeq,
|
||||
endSeq,
|
||||
eventCount: events.length,
|
||||
startedAt: startedAt.toISOString(),
|
||||
endedAt: endedAt.toISOString(),
|
||||
bytesUncompressed,
|
||||
bytesCompressed: gzipped.length,
|
||||
sha256,
|
||||
firstEventHash,
|
||||
lastEventHash,
|
||||
}
|
||||
const signature = createHmac('sha256', this.signingKey)
|
||||
.update(JSON.stringify(unsigned))
|
||||
.digest('hex')
|
||||
const manifest: ArchiveManifest = {
|
||||
...unsigned,
|
||||
signature,
|
||||
sigAlg: 'HMAC-SHA-256',
|
||||
}
|
||||
const manifestJson = JSON.stringify(manifest, null, 2)
|
||||
const manifestSha256 = createHash('sha256').update(manifestJson).digest('hex')
|
||||
|
||||
try {
|
||||
await this.cold.put({
|
||||
key: jsonlKey,
|
||||
body: gzipped,
|
||||
contentType: 'application/gzip',
|
||||
serverSideEncryption: this.serverSideEncryption,
|
||||
})
|
||||
await this.cold.put({
|
||||
key: manifestKey,
|
||||
body: manifestJson,
|
||||
contentType: 'application/json',
|
||||
serverSideEncryption: this.serverSideEncryption,
|
||||
})
|
||||
|
||||
// Confirm both objects landed before deleting from hot.
|
||||
const [jsonlHead, manifestHead] = await Promise.all([
|
||||
this.cold.head(jsonlKey),
|
||||
this.cold.head(manifestKey),
|
||||
])
|
||||
if (!jsonlHead.exists || !manifestHead.exists) {
|
||||
throw new Error(
|
||||
`archive HEAD failed: jsonl=${jsonlHead.exists} manifest=${manifestHead.exists}`,
|
||||
)
|
||||
}
|
||||
} catch (err) {
|
||||
this.logger.error(
|
||||
`archive upload failed (seq ${startSeq}-${endSeq}): ${
|
||||
err instanceof Error ? err.message : String(err)
|
||||
} — events remain in hot Mongo`,
|
||||
)
|
||||
return {
|
||||
ok: false,
|
||||
reason: err instanceof Error ? err.message : String(err),
|
||||
startSeq,
|
||||
endSeq,
|
||||
eventCount: events.length,
|
||||
}
|
||||
}
|
||||
|
||||
// Record the batch BEFORE the delete so a crash between the two leaves
|
||||
// us with a batch record pointing at events that still exist in hot —
|
||||
// safe. The reverse (delete-then-record) could orphan events.
|
||||
const batch = await this.batches.create({
|
||||
archivedAt: endedAt,
|
||||
startSeq,
|
||||
endSeq,
|
||||
eventCount: events.length,
|
||||
manifestSha256,
|
||||
jsonlKey,
|
||||
manifestKey,
|
||||
bytesUncompressed,
|
||||
})
|
||||
|
||||
// Delete from hot. One bulk op; safe because we own this seq range.
|
||||
await this.events
|
||||
.deleteMany({ chained: true, seq: { $gte: startSeq, $lte: endSeq } })
|
||||
.exec()
|
||||
|
||||
this.logger.log(
|
||||
`archive batch ${batch._id} · seq ${startSeq}-${endSeq} · ${events.length} events → ${jsonlKey}`,
|
||||
)
|
||||
|
||||
return {
|
||||
ok: true,
|
||||
batchId: String(batch._id),
|
||||
startSeq,
|
||||
endSeq,
|
||||
eventCount: events.length,
|
||||
}
|
||||
}
|
||||
|
||||
listBatches(limit = 100): Promise<ArchiveBatchDocument[]> {
|
||||
return this.batches.find().sort({ archivedAt: -1 }).limit(limit).exec()
|
||||
}
|
||||
|
||||
// Oldest chained event still in hot Mongo. Used by /audit/verify to report
|
||||
// the tier boundary.
|
||||
async oldestHotSeq(): Promise<number | null> {
|
||||
const e = await this.events.findOne({ chained: true }, { seq: 1 }).sort({ seq: 1 }).exec()
|
||||
return e?.seq ?? null
|
||||
}
|
||||
|
||||
// Highest seq archived (across all batches).
|
||||
async highestArchivedSeq(): Promise<number | null> {
|
||||
const b = await this.batches.findOne().sort({ endSeq: -1 }).exec()
|
||||
return b?.endSeq ?? null
|
||||
}
|
||||
}
|
||||
|
||||
// Removes Mongoose-internal keys we don't want in the cold JSONL.
|
||||
// _id stays (it's a useful retrieval handle for compliance lookups), but
|
||||
// __v and any virtual aliases are stripped.
|
||||
function stripMongoMeta(doc: Record<string, unknown>): Record<string, unknown> {
|
||||
const out: Record<string, unknown> = {}
|
||||
for (const [k, v] of Object.entries(doc)) {
|
||||
if (k === '__v') continue
|
||||
out[k] = v
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// S3 key layout: audit/YYYY/MM/DD/seq-{start}-{end}. Year/month/day buckets
|
||||
// keep S3 list responses manageable as years roll over.
|
||||
function objectKeyPrefix(at: Date, startSeq: number, endSeq: number): string {
|
||||
const y = at.getUTCFullYear()
|
||||
const m = String(at.getUTCMonth() + 1).padStart(2, '0')
|
||||
const d = String(at.getUTCDate()).padStart(2, '0')
|
||||
return `audit/${y}/${m}/${d}/seq-${startSeq}-${endSeq}`
|
||||
}
|
||||
@@ -0,0 +1,78 @@
|
||||
// Daily-ish scheduler for the audit archive run. Disabled by default
|
||||
// (ARCHIVE_ENABLED=false) — production turns it on once volumes warrant.
|
||||
// The operator UI can force a run via POST /audit/archive/run regardless
|
||||
// of this flag.
|
||||
|
||||
import {
|
||||
Injectable,
|
||||
Logger,
|
||||
type OnApplicationBootstrap,
|
||||
type OnModuleDestroy,
|
||||
} from '@nestjs/common'
|
||||
import { ConfigService } from '@nestjs/config'
|
||||
import { ArchiveService } from './archive.service.js'
|
||||
|
||||
// One hour. Inside each tick we check if the current UTC hour matches the
|
||||
// configured run-hour; only one of 24 ticks per day actually invokes the
|
||||
// archive. Cheap and simple — no real cron lib needed.
|
||||
const TICK_MS = 60 * 60 * 1000
|
||||
const DEFAULT_RUN_HOUR_UTC = 3 // 03:00 UTC daily
|
||||
|
||||
@Injectable()
|
||||
export class ArchiveWorker implements OnApplicationBootstrap, OnModuleDestroy {
|
||||
private readonly logger = new Logger(ArchiveWorker.name)
|
||||
private readonly enabled: boolean
|
||||
private readonly runHour: number
|
||||
private timer: NodeJS.Timeout | null = null
|
||||
private lastRunDay: string | null = null
|
||||
|
||||
constructor(
|
||||
private readonly archive: ArchiveService,
|
||||
config: ConfigService,
|
||||
) {
|
||||
this.enabled = config.get('ARCHIVE_ENABLED') === 'true'
|
||||
this.runHour = Number(config.get('ARCHIVE_RUN_HOUR_UTC') ?? DEFAULT_RUN_HOUR_UTC)
|
||||
}
|
||||
|
||||
onApplicationBootstrap(): void {
|
||||
if (!this.enabled) {
|
||||
this.logger.log(
|
||||
'ARCHIVE_ENABLED=false — archive scheduler dormant. Operator UI can force runs.',
|
||||
)
|
||||
return
|
||||
}
|
||||
this.logger.log(`Archive scheduler active · runs daily at ${this.runHour}:00 UTC`)
|
||||
// Fire once on startup in case we missed a window; the day-guard prevents
|
||||
// double-runs within the same UTC day.
|
||||
void this.tick()
|
||||
this.timer = setInterval(() => void this.tick(), TICK_MS)
|
||||
}
|
||||
|
||||
onModuleDestroy(): void {
|
||||
if (this.timer) clearInterval(this.timer)
|
||||
}
|
||||
|
||||
private async tick(): Promise<void> {
|
||||
const now = new Date()
|
||||
if (now.getUTCHours() !== this.runHour) return
|
||||
const today = now.toISOString().slice(0, 10) // YYYY-MM-DD UTC
|
||||
if (this.lastRunDay === today) return // already ran today
|
||||
this.lastRunDay = today
|
||||
|
||||
this.logger.log(`Archive tick fired for ${today}`)
|
||||
try {
|
||||
const res = await this.archive.runOnce()
|
||||
if (res.ok && res.eventCount) {
|
||||
this.logger.log(`Archive complete: ${res.eventCount} events (${res.startSeq}-${res.endSeq})`)
|
||||
} else if (res.ok) {
|
||||
this.logger.log(`Archive complete: ${res.reason ?? 'no-op'}`)
|
||||
} else {
|
||||
this.logger.error(`Archive failed: ${res.reason}`)
|
||||
}
|
||||
} catch (err) {
|
||||
this.logger.error(
|
||||
`Archive tick crashed: ${err instanceof Error ? err.message : String(err)}`,
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,110 @@
|
||||
// Thin wrapper around @aws-sdk/client-s3 configured for Dezky's audit
|
||||
// archive bucket. Same code talks to MinIO in dev and Hetzner Object
|
||||
// Storage in prod — only the endpoint + credentials change via env.
|
||||
//
|
||||
// We deliberately keep the surface small (put, head, list) so the archive
|
||||
// worker doesn't depend on AWS SDK specifics that might shift between
|
||||
// versions. Streaming reads / restore-from-cold are a follow-up; today's
|
||||
// scope is the write path.
|
||||
|
||||
import { Injectable, Logger } from '@nestjs/common'
|
||||
import { ConfigService } from '@nestjs/config'
|
||||
import {
|
||||
HeadObjectCommand,
|
||||
ListObjectsV2Command,
|
||||
PutObjectCommand,
|
||||
S3Client,
|
||||
} from '@aws-sdk/client-s3'
|
||||
|
||||
export interface ColdPutInput {
|
||||
key: string
|
||||
body: Buffer | string
|
||||
contentType?: string
|
||||
// Server-side encryption for prod (Hetzner SSE-S3). MinIO ignores when
|
||||
// unset; setting it explicitly is a no-op for dev but documents intent.
|
||||
serverSideEncryption?: 'AES256'
|
||||
}
|
||||
|
||||
export interface ColdHeadResult {
|
||||
exists: boolean
|
||||
size?: number
|
||||
etag?: string
|
||||
}
|
||||
|
||||
@Injectable()
|
||||
export class ColdStorageClient {
|
||||
private readonly logger = new Logger(ColdStorageClient.name)
|
||||
private readonly client: S3Client
|
||||
private readonly bucket: string
|
||||
|
||||
constructor(config: ConfigService) {
|
||||
const endpoint = config.getOrThrow<string>('AUDIT_COLD_ENDPOINT')
|
||||
const region = config.get<string>('AUDIT_COLD_REGION') ?? 'us-east-1'
|
||||
const accessKeyId = config.getOrThrow<string>('AUDIT_COLD_ACCESS_KEY')
|
||||
const secretAccessKey = config.getOrThrow<string>('AUDIT_COLD_SECRET_KEY')
|
||||
this.bucket = config.getOrThrow<string>('AUDIT_COLD_BUCKET')
|
||||
|
||||
this.client = new S3Client({
|
||||
endpoint,
|
||||
region,
|
||||
credentials: { accessKeyId, secretAccessKey },
|
||||
// forcePathStyle is mandatory for MinIO and harmless for Hetzner (which
|
||||
// accepts both virtual-host + path-style). Without it the SDK tries
|
||||
// `bucket.minio:9000` which doesn't resolve.
|
||||
forcePathStyle: true,
|
||||
})
|
||||
}
|
||||
|
||||
get bucketName(): string {
|
||||
return this.bucket
|
||||
}
|
||||
|
||||
// Upload an object. Returns the ETag the server assigned. Throws on
|
||||
// network/auth failure; callers should not catch and pretend success —
|
||||
// the archive worker uses throw-on-failure to keep events in hot storage.
|
||||
async put(input: ColdPutInput): Promise<{ etag?: string }> {
|
||||
const res = await this.client.send(
|
||||
new PutObjectCommand({
|
||||
Bucket: this.bucket,
|
||||
Key: input.key,
|
||||
Body: input.body,
|
||||
ContentType: input.contentType,
|
||||
ServerSideEncryption: input.serverSideEncryption,
|
||||
}),
|
||||
)
|
||||
return { etag: res.ETag }
|
||||
}
|
||||
|
||||
// Existence + size check. The archive worker calls this after every put
|
||||
// to confirm the upload actually landed before deleting from hot Mongo.
|
||||
async head(key: string): Promise<ColdHeadResult> {
|
||||
try {
|
||||
const res = await this.client.send(
|
||||
new HeadObjectCommand({ Bucket: this.bucket, Key: key }),
|
||||
)
|
||||
return { exists: true, size: res.ContentLength, etag: res.ETag }
|
||||
} catch (err) {
|
||||
// S3 returns 404 for missing keys; the SDK surfaces it as a NotFound
|
||||
// error. Treat 404 as "doesn't exist" without logging — anything else
|
||||
// is real failure.
|
||||
const e = err as { name?: string; $metadata?: { httpStatusCode?: number } }
|
||||
if (e?.name === 'NotFound' || e?.$metadata?.httpStatusCode === 404) {
|
||||
return { exists: false }
|
||||
}
|
||||
throw err
|
||||
}
|
||||
}
|
||||
|
||||
// List objects under a prefix. Used by the operator UI to show recent
|
||||
// archive batches. Paginated via continuation token; for the v1 UI we cap
|
||||
// at a single page since we only show the latest N.
|
||||
async list(prefix: string, max = 100): Promise<{ keys: string[]; truncated: boolean }> {
|
||||
const res = await this.client.send(
|
||||
new ListObjectsV2Command({ Bucket: this.bucket, Prefix: prefix, MaxKeys: max }),
|
||||
)
|
||||
return {
|
||||
keys: (res.Contents ?? []).map((c) => c.Key!).filter(Boolean),
|
||||
truncated: !!res.IsTruncated,
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
import { Module } from '@nestjs/common'
|
||||
import { MongooseModule } from '@nestjs/mongoose'
|
||||
import {
|
||||
ArchiveBatch,
|
||||
ArchiveBatchSchema,
|
||||
} from '../schemas/archive-batch.schema.js'
|
||||
import { AuditEvent, AuditEventSchema } from '../schemas/audit-event.schema.js'
|
||||
import { ArchiveService } from './archive.service.js'
|
||||
import { ArchiveWorker } from './archive.worker.js'
|
||||
import { ColdStorageClient } from './cold-storage.client.js'
|
||||
|
||||
// Phase 4 — cold storage + retention. Owns the S3 client wrapper, the
|
||||
// archive service that moves events from hot Mongo to cold S3, and the
|
||||
// daily worker scheduler. AuditModule imports this and exposes archive
|
||||
// endpoints on /audit/archive*.
|
||||
@Module({
|
||||
imports: [
|
||||
MongooseModule.forFeature([
|
||||
{ name: AuditEvent.name, schema: AuditEventSchema },
|
||||
{ name: ArchiveBatch.name, schema: ArchiveBatchSchema },
|
||||
]),
|
||||
],
|
||||
providers: [ColdStorageClient, ArchiveService, ArchiveWorker],
|
||||
exports: [ArchiveService],
|
||||
})
|
||||
export class ColdModule {}
|
||||
@@ -0,0 +1,50 @@
|
||||
import { Prop, Schema, SchemaFactory } from '@nestjs/mongoose'
|
||||
import { HydratedDocument } from 'mongoose'
|
||||
|
||||
export type ArchiveBatchDocument = HydratedDocument<ArchiveBatch>
|
||||
|
||||
// One row per successful archive run. Each row points at the S3 objects
|
||||
// (JSONL + manifest) and records the manifest's sha256 + signature so the
|
||||
// operator can verify cold-tier integrity without downloading the data.
|
||||
//
|
||||
// We keep these forever (negligible storage; one row per day at most). The
|
||||
// operator UI lists them with timestamps + seq ranges; offline tooling can
|
||||
// fetch the JSONL by `objectKey` for compliance retrieval.
|
||||
@Schema({ collection: 'archive_batches', timestamps: true })
|
||||
export class ArchiveBatch {
|
||||
// ISO date of the archive run (when we moved data, not when the events
|
||||
// happened). Used to bucket UI by day.
|
||||
@Prop({ required: true, index: true })
|
||||
archivedAt!: Date
|
||||
|
||||
// Inclusive seq range of events archived in this batch. The next batch
|
||||
// starts at `endSeq + 1`.
|
||||
@Prop({ required: true, index: true })
|
||||
startSeq!: number
|
||||
|
||||
@Prop({ required: true })
|
||||
endSeq!: number
|
||||
|
||||
@Prop({ required: true })
|
||||
eventCount!: number
|
||||
|
||||
// sha256 of the manifest JSON (after signing). The full manifest is in S3;
|
||||
// storing the hash here lets us detect manifest-level tampering without
|
||||
// downloading anything.
|
||||
@Prop({ required: true })
|
||||
manifestSha256!: string
|
||||
|
||||
// S3 object keys (Bucket is global from ColdStorageClient).
|
||||
@Prop({ required: true })
|
||||
jsonlKey!: string
|
||||
|
||||
@Prop({ required: true })
|
||||
manifestKey!: string
|
||||
|
||||
// Uncompressed event payload bytes (informational; the gzip'd object size
|
||||
// lives in S3 metadata).
|
||||
@Prop({ required: true })
|
||||
bytesUncompressed!: number
|
||||
}
|
||||
|
||||
export const ArchiveBatchSchema = SchemaFactory.createForClass(ArchiveBatch)
|
||||
Reference in New Issue
Block a user