Security architecture¶
The trust boundaries of the system and the control the code applies at each one. This is a description of mechanisms. The threat model, attack scenarios and open findings are in docs/security/SECURITY_AUDIT.md; the baseline findings (F-01 to F-21) are in docs/audit/INITIAL_AUDIT.md.
1. Trust boundaries¶
flowchart LR
subgraph Internet["Untrusted: Internet"]
Client["API clients"]
Browser["Browser running the web UI<br/>API key in localStorage"]
Direct["Direct callers of the<br/>Cloud Run URL"]
end
subgraph Edge["Boundary 1: Cloudflare edge"]
Assets["Static UI<br/>CSP from web/public/_headers"]
Worker["Worker<br/>route allowlist, key presence,<br/>64 KiB cap, rate limit"]
end
subgraph Origin["Boundary 2: Cloud Run origin"]
Lock["Origin lock<br/>X-Origin-Secret, compare_digest"]
App["Body cap, authentication, schema,<br/>token budget, admission"]
Engine["llama.cpp engine"]
end
subgraph Third["Third-party services"]
Hub[("Hugging Face Hub")]
Redis[("Upstash Redis, optional")]
PG[("Neon Postgres, optional")]
end
subgraph Build["Boundary 3: build and release"]
GHA["GitHub Actions"]
SM[("Secret Manager")]
end
Client --> Worker
Browser --> Assets
Browser --> Worker
Worker -->|"adds the secret header"| Lock
Direct -->|"no secret: 403"| Lock
Lock --> App --> Engine
Engine -->|"pinned revision, SHA-256 verified"| Hub
Worker -.-> Redis
App -.->|"lookup by key hash"| PG
GHA -->|"WIF, main branch only"| SM
SM -->|"env vars"| App
| Boundary | What crosses it | Controls | Code |
|---|---|---|---|
| Internet to edge | HTTP requests with an API key | Path allowlist (unknown paths 404, /metrics blocked), key presence, 64 KiB body cap, per-key and per-IP rate limit, response header hygiene |
deploy/cloudflare-worker/src/ |
| Internet to origin | Requests that bypass the edge | Cloud Run ingress is all with allUsers as invoker, so the origin secret is the barrier: 403 without it |
api.py request_context, deploy/cloudrun/service.yaml, release.yml |
| Edge to origin | Forwarded requests | Shared secret header; client-supplied x-origin-secret, x-forwarded-for, x-request-id, hop-by-hop and cf-* headers are stripped before forwarding |
src/proxy.ts buildForwardHeaders |
| Origin to model weights | GGUF download | Pinned Hub revision, SHA-256 verified before load, fail closed | hub.py |
| Origin to Postgres | Key lookups, usage, feedback | Keys stored and queried only as SHA-256 hashes; TLS by connection string (sslmode=require in the runbook) |
storage/, auth.py |
| CI to cloud | Deploys, secrets | Workload Identity Federation restricted to this repository's main; no long-lived cloud keys |
deploy/gcp/bootstrap.sh, release.yml |
2. Authentication and authorisation¶
Origin lock¶
- When
ORIGIN_SHARED_SECRETis set, every request except/health,/readyand/metricsmust carryX-Origin-Secretwith the same value, compared withhmac.compare_digest(constant time). Otherwise: 403 before the body is read (api.pyrequest_context, the outermost middleware). - The Worker sets the header from its own secret and drops any value the client sent
(
src/proxy.ts). - The secret is 32 random bytes, hex-encoded (
scripts/setup_secrets.sh). Release refuses to deploy if it is missing (release.yml, secret sync).
API keys¶
| Aspect | Implementation | Code |
|---|---|---|
| Transport | X-API-Key: <key> or Authorization: Bearer <key> |
auth.py extract_api_key, src/identity.ts |
| Edge | Checks presence only (401 when absent). The key is hashed (SHA-256, first 128 bits) to form the rate-limit identity and is never logged or stored in Redis. | index.ts, src/identity.ts |
| Static key | API_KEY compared with hmac.compare_digest; fa_ plus 32 random bytes hex (scripts/setup_secrets.sh) |
auth.py |
| Per-user keys | fa_ plus secrets.token_urlsafe(32) (256 bits). Only sha256(key) is stored (api_keys.key_hash); the plaintext is printed once by fastapi-assistant keys create. |
storage/base.py |
| Lookup cache | In-process TTL cache keyed by the hash: hits for KEY_CACHE_TTL_S (60 s), misses for at most 10 s. Revocation takes effect within the TTL. |
auth.py _TTLCache |
| Off switch | With neither API_KEY nor DATABASE_URL, every caller is anonymous. Production always sets API_KEY (the Release sync fails without it). |
auth.py, release.yml |
Why a fast hash is the right choice here: the keys are 256-bit uniformly random
values, not human-chosen passwords. Recovering a key from its SHA-256 would take on
the order of 2^255 guesses, so a slow password hash (bcrypt, scrypt, Argon2) adds no
practical protection. It would add cost to every request, because the hash is
computed before the TTL cache lookup. CodeQL alert #2
(py/weak-sensitive-data-hashing on storage/base.py hash_api_key) is accepted
for this reason and dismissed as "won't fix" (F-15).
Metrics¶
/metrics is blocked at the edge (404). At the origin it is exempt from the origin
lock so a scraper can reach it directly. When METRICS_TOKEN is set it requires
Authorization: Bearer <token>, compared with hmac.compare_digest (api.py
metrics_endpoint). Release requires the token. Without it, the endpoint would be
open.
3. Browser-facing controls¶
| Control | Value | Code |
|---|---|---|
| CORS | allow_origins from CORS_ALLOW_ORIGINS (default *), allow_credentials=False, methods GET, POST, OPTIONS. Authentication is a header, never a cookie, so a wildcard origin does not enable cross-site request forgery (F-18). |
api.py, config.py |
| Origin response headers | X-Content-Type-Options: nosniff, X-Frame-Options: DENY, Referrer-Policy: no-referrer on every response |
api.py _SECURITY_HEADERS |
| Worker response headers | nosniff on every response it produces or relays; removes x-cloud-trace-context, server, via, alt-svc |
src/http.ts |
| Web UI headers | CSP default-src 'self'; script-src 'self'; style-src 'self'; img-src 'self' data:; connect-src 'self' https: ...; object-src 'none'; base-uri 'none'; frame-ancestors 'none', plus nosniff, X-Frame-Options: DENY, Referrer-Policy, Permissions-Policy, Cross-Origin-Opener-Policy |
web/public/_headers |
| Markdown rendering | react-markdown without a raw-HTML plugin, so model output cannot inject markup; img-src blocks remote images (tracking pixels) |
web/src/components/Markdown.tsx, web/public/_headers |
The UI keeps the API key in localStorage (web/src/state/settings.ts). A script
injected into the page could read it; the CSP (script-src 'self', no inline
script) and escaped Markdown are what prevent such a script.
4. Input and resource limits¶
Each limit stops a class of abuse before it reaches the expensive step.
| Limit | Value | Enforced at | Code |
|---|---|---|---|
| Path allowlist | Unknown paths 404 | Edge | src/routes.ts |
| Request body | 65,536 bytes, declared and streamed | Edge and origin | src/body.ts, limits.py |
| Rate limit | 20 per minute per key, 10 per minute per IP | Edge, Redis or per isolate | src/ratelimit.ts |
| Schema | Up to 100 messages of up to 8,000 characters; max_tokens up to 1,024; question up to 4,000 characters |
Origin | schemas.py |
| Conversation | Up to 6,000 characters (MAX_INPUT_CHARS), last 6 turns |
Origin | api.py, prompts.py |
| System prompt | Client system messages are dropped; the service's own prompt is always used | Origin | prompts.py normalize_chat |
| Prompt tokens | Must leave 16 tokens of the 2,048-token context; answer clamped to the room left | Origin, before admission | engines/base.py fit |
| Concurrency | 1 generation per instance, 3 waiting, 60 s queue timeout | Origin | admission.py, service.yaml |
| Generation time | 60 s per generation on Cloud Run (120 s default), checked per token; queue plus generation stays under the edge's 150 s timeout | Origin | engines/base.py Deadline, service.yaml |
| Spend | maxScale 2 by default; budget alert notifies only |
Cloud Run | service.yaml, bootstrap.sh |
Client-facing error messages are fixed strings ("Generation failed", "Model not
ready"). Load errors are logged server-side and never returned (api.py ready).
5. Secrets¶
| Secret | Generated by | Stored in | Used by |
|---|---|---|---|
ORIGIN_SHARED_SECRET |
scripts/setup_secrets.sh (32 random bytes) |
Worker secret, GitHub secret, Secret Manager | Edge and origin |
API_KEY |
scripts/setup_secrets.sh (32 random bytes) |
GitHub secret, Secret Manager | Origin |
METRICS_TOKEN |
scripts/setup_secrets.sh (24 random bytes) |
GitHub secret, Secret Manager, a private local file | Origin, metrics scraper |
| Upstash URL and token | Upstash console (optional) | Worker secrets, GitHub secrets | Edge |
DATABASE_URL, SENTRY_DSN |
Neon, Sentry (optional) | GitHub secrets, Secret Manager (disabled placeholder when absent) |
Origin |
HF_TOKEN |
Hugging Face | Kaggle kernel secret | Training pipeline only |
No secret appears in wrangler.toml, service.yaml or any tracked file; the
service spec references Secret Manager entries by name. The runtime service account
can only read secrets (roles/secretmanager.secretAccessor).
6. Model artifact integrity¶
model-release.jsonpins the GGUF by repository, file name, Hub commit and SHA-256.hub.resolve_gguf_pathdownloads at that revision and callsverify_sha256, which raises on a mismatch. The load state becomesfailed,/readystays 503, and the Cloud Run startup probe never passes, so a substituted or corrupted file is never served.- Verification is skipped only when
GGUF_SHA256is unset;config.pyrejects a value that is not 64 hex characters. - The training pipeline records the SHA-256 of the file it publishes
(
kaggle/pipeline.py) and tags each Hub release withexist_ok=False, so the tooling never moves an existing tag (export.pypush_folder).
7. Supply chain¶
| Control | Where | Effect |
|---|---|---|
| Hash-pinned serving lock | requirements/serve.txt (make lock, uv pip compile --generate-hashes); Dockerfile pip wheel --require-hashes |
The image build fails on any artifact whose hash differs |
| Lockfile installs | npm ci in CI and Release for the Worker and the web UI |
Reproducible JavaScript dependencies |
| Pinned Actions | Third-party actions (Docker, Trivy, Google, Cloudflare, gitleaks, k6) pinned to commit SHAs; GitHub-owned actions/* and github/codeql-action by major tag |
A moved third-party tag cannot change the pipeline |
| Least-privilege tokens | Every workflow sets explicit top-level permissions (mostly contents: read); jobs add only what they need, such as id-token: write for the Cloud Run deploy |
|
| Trivy | CI image job and Release scan job |
Fails on fixable HIGH or CRITICAL CVEs in the image |
| CodeQL | .github/workflows/codeql.yml |
Python, JavaScript/TypeScript, Actions |
| gitleaks | CI secrets job, full history (fetch-depth: 0) |
Fails on committed secrets |
| pip-audit | .github/workflows/security.yml on requirements/serve.txt |
Fails on known-vulnerable Python dependencies; one reviewed exception (PYSEC-2026-2447, unreachable code path) |
| npm audit | security.yml, Worker and web |
Fails on any advisory in production dependencies, and on high or critical in any dependency |
| Dependency review | security.yml, pull requests |
Fails a PR that adds a high-severity dependency |
| Dependabot | .github/dependabot.yml |
Weekly updates for pip, npm, Docker, Compose and Actions |
| Minimal runtime image | Dockerfile |
No compiler, no pip, setuptools or wheel; Debian updates applied; uid 10001 |
8. Logging and privacy¶
| Data | Treatment | Code |
|---|---|---|
| API keys | Never logged at the edge or the origin; the edge logs only the identity type (key or ip) |
index.ts, api.py |
| Questions and answers | Stored with usage events only when STORE_TRANSCRIPTS=true (default false) |
service.py record_usage |
| Feedback | Stored in Postgres; without a database, only the rating and the comment length are written to the log, never the comment text | api.py feedback |
| Errors | Sentry only when SENTRY_DSN is a URL, with send_default_pii=False |
api.py _init_sentry |
| Request ids | Client X-Request-ID accepted at the edge only if it matches [A-Za-z0-9._:-]{1,128} |
src/http.ts sanitizeRequestId |
9. Known limits¶
Accepted or open, each traceable to code:
- Key presence is not key validity at the edge. Any non-empty key string passes
the edge, and each distinct string gets its own per-key rate-limit bucket
(
index.ts, identity selection). The origin rejects unknown keys at authentication, after the body cap and JSON decode and before any model work. - The fallback rate limiter is per isolate. Without Upstash, each Cloudflare
isolate counts separately, so the effective limit is a multiple of the configured
one (
src/ratelimit.ts). See DECISIONS.md, ADR-008. This fallback is the rate limit that is live today (docs/IMPLEMENTATION_STATUS.md). - Probes are public and report backend and model version (F-16, accepted).
- CORS allows any origin (F-18, accepted; header authentication).
- Controls that switch off when unset: origin lock (
ORIGIN_SHARED_SECRET), authentication (API_KEY,DATABASE_URL), metrics token (METRICS_TOKEN) and weight verification (GGUF_SHA256). Release requires the first three for Cloud Run, andmodel-release.jsoncarries the fourth.