Use SCRUBR as an internal LLM gateway

Run one SCRUBR deployment as an internal endpoint and point every product at it instead of the LLM providers. SCRUBR masks secrets and PII on the way out, forwards to the real provider (or your own model gateway), and rehydrates the response on the way back — so the provider only ever sees placeholders and your apps still get the real answer. One gateway can front many providers.

flowchart LR
    A1["service A"] --> G
    A2["service B"] --> G
    A3["batch jobs"] --> G
    G{"SCRUBR<br/>internal gateway"}
    G -->|"/openai"| P1[["OpenAI"]]
    G -->|"/anthropic"| P2[["Anthropic"]]
    G -->|"/gemini"| P3[["Gemini"]]
    G -->|"/gateway"| P4[["your model proxy"]]

Everything below uses the reverse-proxy mode — change a base URL. No CA, no client SDK changes, no TLS interception.

1. Write the gateway config

One config, several routes: front as many providers (and your own gateway) as you like. Each route names a profile that says which JSON paths to scan.

# scrubr.yaml — internal gateway
auth:
  enabled: true                 # only callers presenting the key may use the gateway
  header: x-scrubr-key
  keys: ["REPLACE_WITH_A_LONG_RANDOM_KEY"]   # rotate; treat as a shared secret

routes:
  - { listen_path: /openai,    upstream: https://api.openai.com,                    profile: openai }
  - { listen_path: /anthropic, upstream: https://api.anthropic.com,                 profile: anthropic }
  - { listen_path: /gemini,    upstream: https://generativelanguage.googleapis.com, profile: gemini }
  # Front your own OpenAI-compatible model proxy too:
  - { listen_path: /gateway,   upstream: https://models.internal,                   profile: openai }

profiles:
  openai:    { scan_paths: ["messages[].content"],           stream_paths: ["choices[].delta.content"] }
  anthropic: { scan_paths: ["messages[].content", "system"], stream_paths: ["delta.text"] }
  gemini:    { scan_paths: ["contents[].parts[].text"],      stream_paths: ["candidates[].content.parts[].text"] }

masking:
  mode: dry-run                 # validate coverage first, then switch to enforce

rules:
  - { name: email, type: EMAIL, pattern: '[\w.+-]+@[\w.-]+\.\w+', priority: 50 }
  # Add the curated token/key/credential ruleset from examples/common-rules.yaml.

Start in dry-run: SCRUBR reports what it would mask (via the x-scrubr-detected response header and the audit log) but forwards the original, so you can confirm coverage without risk. Switch to enforce once you trust it.

2. Deploy it — internal only

Use the Helm chart and keep the Service in-cluster (the default ClusterIP):

helm install scrubr oci://ghcr.io/scrubr-dev/charts/scrubr --version X.Y.Z -f my-values.yaml

# my-values.yaml — the `config:` block becomes the mounted scrubr.yaml
config:
  # ... paste the gateway config from step 1 here ...
metrics:
  enabled: true                 # Prometheus /metrics on a separate admin port

The pods run non-root on a read-only rootfs, expose /healthz + /readyz probes, and serve :8080. For multiple replicas sharing session-scoped pseudonyms across a conversation, follow Deploy on Kubernetes → High availability (StatefulSet + Redis, encrypted at rest).

Keep it private. Expose the Service only inside the cluster/VPC and restrict it with a NetworkPolicy. Do not put it on the public internet unless you also front it with TLS, authentication, and rate limiting.

3. Point your products at it

Swap the base URL and add the gateway key. Your app still sends the provider's own credential — SCRUBR forwards it upstream and never stores it; it only strips its own x-scrubr-key header.

OpenAI SDK (Python):

from openai import OpenAI
client = OpenAI(
    base_url="http://scrubr.<namespace>.svc:8080/openai",   # was https://api.openai.com/v1
    api_key="<your OpenAI key>",                             # forwarded upstream by SCRUBR
    default_headers={"x-scrubr-key": "<gateway key>"},
)

curl:

curl -sS http://scrubr.<namespace>.svc:8080/openai/v1/chat/completions \
  -H "x-scrubr-key: <gateway key>" \
  -H "Authorization: Bearer <your OpenAI key>" \
  -H "content-type: application/json" \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"email me at a@b.com"}]}' \
  -D - | grep -i x-scrubr

You'll see x-scrubr-mode and x-scrubr-detected response headers; the provider received the email masked as ⟦S:EMAIL·…⟧, and the reply comes back rehydrated.

Want to centralize provider keys so individual services don't hold them? Point a SCRUBR route at a credential-injecting gateway you already run (/gateway above): service → SCRUBR (masks) → your gateway (adds the provider key) → provider. SCRUBR forwards, but does not inject, upstream credentials.

4. Multi-turn conversations

For stable pseudonyms across a conversation — the same secret maps to the same sentinel on every turn — enable session scope: set masking.scope: session, have callers send a per-conversation x-scrubr-session header (an unguessable, one-per-user value), and run the HA setup so the session map is shared through Redis. See the configuration reference and the HA guide.

5. Observe and verify

Health: /healthz (liveness) and /readyz (readiness) on the proxy port.
Metrics: enable metrics.enabled for a Prometheus /metrics endpoint (requests, detections, and latency by route). See Observability.
Audit: turn on the tamper-evident audit log — and optionally the transaction log — to prove exactly what left the boundary masked.

Security checklist

Private by default — ClusterIP + NetworkPolicy; public exposure requires TLS, authentication, and rate limiting.
Gateway auth — keep auth.enabled and rotate the x-scrubr-key.
TLS — terminate at your ingress/mesh, or in SCRUBR (tls.enabled).
Provider keys stay with callers — SCRUBR forwards them upstream; it never stores them.
Session keys are bearer secrets — one per user, unguessable.
Roll out in dry-run, confirm coverage on the x-scrubr-detected header and the audit log, then switch to enforce.