One API Key, Many Safety Surfaces: A Structured Model Architecture

The operational constraint is publication, not inference: every surface needs a defensible answer about whether content may become visible. Short answer: put text and image moderation behind one server-side policy gateway, keep the API key there, and make a versioned decision record - rather than a model response - the contract used by chat, comments, avatars, and marketplace uploads. That distinction matters. A single credential can simplify access, but it can't make four product surfaces share the same latency budget, release rule, or consequence for uncertainty. The smallest architecture worth shipping has one input envelope, one structured evidence shape, surface-specific policy, and explicit pending states. Start there. How should text and image moderation cover chat, comments, avatars, and uploads? Treat the moderation gateway as a narrow internal boundary. Product handlers send it an immutable content reference, a content hash, the surface, the media kind, and the current policy version. The gateway owns authentication and converts an external or self-hosted check into a stable evidence record. A policy function then maps that evidence to an application action such as allow , hold , or reject . Credentials never reach a browser or mobile client. The important split is between evidence and action. A model can return category signals in structured output, but those signals don't know whether the item is a private message, a public comment, an avatar, or a seller's listing photo. The application does. A borderline signal might place a public upload in review while a different surface follows a different, evaluated rule. That isn't inconsistency; it is policy expressed where the business context exists. Calling the checker directly from every request handler looks shorter in a notebook. It also copies schema parsing, credential access, timeout behavior, and policy mapping into unrelated code paths. Once those copies drift, a policy replay no longer answers the question you care about: “What would the current rules do to the same accepted bytes?” A gateway avoids that drift without pretending every request must be synchronous. One API key is therefore an implementation detail, not the architecture. Keep it in a secret store available only to the adapter, rotate it without changing product clients, and attribute each request to an internal surface and resource. If different modalities or residency requirements later demand separate checkers, the stable envelope and decision record can remain in place while the adapter changes underneath. Make the decision record replayable I want the first notebook experiment to produce the same kind of artifact that the production replay harness will inspect. A Boolean flagged field is too weak. It loses the policy version, reason codes, input identity, and the distinction between “the checker found no signal” and “no valid decision exists.” Those omissions show up later as irreproducible appeals and misleading evals. A compact contract is enough. Store the resource ID and content hash, not an arbitrary mutable URL. Record the surface and media kind. Keep the checker evidence separate from the final action, attach a policy version, and give each attempt its own identifier. Downstream publication code should read the action and decision state, never raw vendor categories. Here is a focused Python shape. The protocol deliberately says nothing about a particular model or service, so a notebook stub, a hosted adapter, and an internally served checker can all feed the same policy function. from dataclasses import dataclass from enum import StrEnum from typing import Protocol class Action(StrEnum): ALLOW = "allow" HOLD = "hold" REJECT = "reject" @dataclass(frozen=True) class ContentEnvelope: resource_id: str content_hash: str surface: str media_kind: str policy_version: str @dataclass(frozen=True) class Evidence: reason_codes: tuple[str, ...] scores: dict[str, float] @dataclass(frozen=True) class Decision: envelope: ContentEnvelope action: Action reason_codes: tuple[str, ...] class Checker(Protocol): def inspect(self, envelope: ContentEnvelope) -> Evidence: ... def decide(checker: Checker, envelope: ContentEnvelope) -> Decision: evidence = checker.inspect(envelope) action = Action.HOLD if evidence.reason_codes else Action.ALLOW return Decision( envelope=envelope, action=action, reason_codes=evidence.reason_codes, ) The mapping is intentionally conservative sample code, not a universal safety policy. Real mappings belong in a versioned rule table backed by labeled fixtures for each surface. The adapter should reject malformed structured output instead of filling missing fields with guesses. A missing decision is an operational state, not permission to publish. Idempotency belongs in this contract too. Use the resource identity, accepted-content hash, and policy version as the logical moderation key. A worker may retry an attempt, but it should converge on one current decision for that exact content and policy. If an avatar is replaced, its hash changes and it earns a new decision; if the same job is delivered twice, it doesn't create two review cases. This is the notebook-to-prod move I care about most: preserve the question being evaluated. A quick experiment that returns attractive JSON but can't be replayed against immutable inputs has proved syntax, not an operating model. How can delivery paths differ without splitting the policy boundary? Comments and chat messages often arrive as small text requests. Image uploads have a different ingestion problem: the server must accept bytes, validate the allowed format and size, decode them under resource limits, and create the normalized representation that will actually be inspected. The public Sharp documentation is a useful catalogue of image-processing operations for Node pipelines. In a Python service I would choose an appropriate Python image stack, but the architectural rule stays the same - moderation receives a server-accepted artifact, not a client-declared type or an untrusted remote location. Uploads naturally fit a state machine. Create the resource as pending , persist its immutable identity, enqueue moderation, and move it to a visible or review state only after policy consumes a valid decision. Keep the original away from public delivery while it is pending. The UI can show progress, but it can't promote the asset by declaring success locally. Fast text feels different. Some chat designs moderate before fan-out; others let the sender see a private pending representation while public delivery waits. Server-Sent Events can carry status changes from the server to a browser over a one-way connection using EventSource , as MDN documents. SSE changes notification delivery, not authority: the server still owns the publication transition, and clients should be able to reconnect and fetch the current state rather than infer it from a missed event. Don't hide the state. The simple synchronous path is attractive when its measured latency fits the product budget and the content can remain unpublished until the response arrives. A queue is a better fit when decoding, inspection, or human review can exceed that budget. Both paths can call the same adapter and policy code. Forcing them into one transport merely to preserve a one-box diagram creates coupling where none is useful. | Concern | Synchronous request | Queued job | |---|---|---| | Best fit | Small inputs within the measured request budget | Decode-heavy media or review workflows | | Publication | Wait for a valid decision | Remain pending until a valid decision | | Retry identity | Resource, hash, and policy version | Resource, hash, and policy version | | Client update | Normal response | State fetch or server event | Failure handling also needs an explicit rule. Invalid media should stop at ingestion. Invalid structured output should stop at the adapter. A timed-out or interrupted attempt should leave the resource pending and schedule an idempotent retry or review according to policy; it must not silently become allow . Log enough metadata to distinguish those paths without placing sensitive user content in general application logs. Let evals choose thresholds and topology The evaluation set should be organized around product decisions, not generic model categories. Build it from content you are permitted to use, label the expected application action, and retain the rationale. Include ordinary content that resembles prohibited material, quoted or reclaimed language, multilingual examples relevant to the audience, screenshots containing text, difficult crops, duplicate images, and media that fails ingestion. Split results by chat, comments, avatars, and listings because an aggregate score can hide a costly false-reject pattern on one surface. Then measure the whole policy path. I track false allows, false rejects, hold rate, reviewer disagreement, schema-valid response rate, end-to-end latency, pending age, duplicate-job rate, and cost per decided item. For a chat model used as a classifier, I also track input and output tokens by policy version; prompt growth is an architecture change when it affects latency and spend. Request counts alone won't expose repeated context or retries. The experiment sequence is straightforward, but this is where I spend the most care. Freeze an evaluation snapshot, run candidate checkers through the adapter, and apply the exact policy mapping intended for production. Compare final actions, not just model scores. For every disagreement, inspect the accepted input representation, parsed evidence, policy version, and expected action together; otherwise a decoding difference can masquerade as a model difference, or a rule-table change can look like a classifier regression. Next, shadow the candidate on live-shaped traffic that may legally be evaluated without letting it control publication. Replay the frozen fixtures after any prompt, model, prepr