The compliance questions legal will ask before you ship an LLM feature (and how to answer them)

Published December 9, 2024

You built the feature. It works in the demo. Then someone says "run it by legal" and the timeline slips two weeks, because you walk into that meeting with a working prototype and no answers to questions you didn't know you'd be asked.

This is a translation layer. Not legal advice — I'm an engineer, not a lawyer, and none of this replaces your actual counsel. But I've sat in enough of these meetings to know the questions repeat. If you bring answers instead of "let me check," the meeting is thirty minutes instead of three.

Where does the data go

This is always question one. "We use OpenAI" or "we use Claude" is not an answer — legal wants to know the actual data flow.

What to bring:

• Which API you're calling, and which region it processes in. Most major providers let you pick a region or a specific endpoint (e.g., Azure OpenAI's regional deployments, Anthropic's regional options where available, AWS Bedrock's region selection). If you haven't picked one deliberately, that's the first fix.
• The subprocessor list. Every foundation model vendor publishes one — it names cloud infra providers, monitoring tools, anyone else who touches the data in transit. Legal will ask for this list by name because it usually needs to go into your own subprocessor disclosures to customers.
• Whether the request ever leaves your compliance boundary — for example, if you're in a HIPAA or GDPR context, does the API call cross into a jurisdiction or vendor that isn't covered by your existing agreements.

If you're calling a third-party API directly from application code with no gateway in front of it, say so. That's a normal starting point, but it means data residency is whatever the vendor's default is, and you should know that default before the meeting, not discover it in it.

Is it retained, and for how long

Two separate questions get conflated here: retention for abuse monitoring, and retention for model training. They have different answers and different contract terms.

Retention for abuse/safety monitoring. Most providers retain API inputs and outputs for some window (often 30 days, sometimes less under zero-retention agreements) to run safety and abuse detection. This is usually non-negotiable at the API level but can be reduced or waived under enterprise agreements — ask your vendor rep directly whether a zero-data-retention (ZDR) agreement is available for your tier, and get the actual retention window in writing rather than repeating whatever you remember from a pricing page.

Retention for your own logs. Separate from the vendor. If you log prompts and completions for debugging or analytics, that's your retention policy, and it's usually the bigger risk — vendor retention is contractually bounded; your own debug logs sitting in a S3 bucket with no lifecycle policy are not.

What to bring: your vendor's stated retention window (cite the actual DPA section, not "I think it's 30 days"), and a description of what you log on your own side and for how long.

Is it used for training

This is the one legal cares about most, because "our customer's data trained a competitor's feature" is the nightmare scenario that makes headlines.

As of late 2024, the major API providers (OpenAI, Anthropic, Google) all offer contractual no-training guarantees on their standard API products — this is different from their consumer chat products, which historically had opt-out-based training use. The distinction that trips people up: a consumer ChatGPT or Claude.ai account does not carry the same guarantees as the API. If any part of your pipeline uses a consumer-tier account (a common shortcut in early prototypes), flag it — that's very likely a compliance issue waiting to be found.

What to bring:

• The specific clause in your provider's Data Processing Addendum (DPA) that states API inputs/outputs are not used for model training, by default, unless you explicitly opt in.
• Confirmation of which product tier you're actually using in production (API vs. consumer product) — check this even if you're sure, because someone always prototyped with a personal account at some point.
• Whether you've opted into any feedback/improvement programs — some providers offer opt-in programs where you can consent to have specific data used for training, usually for the discount. If your team enrolled without visibility into what "improve our services" meant in the checkbox, find out now.

What about PII

Legal's real question here isn't "do you send PII" — assume the answer is often yes — it's "did you think about it, and what did you do."

Minimization. Before the API call, strip or reduce whatever isn't needed for the model to do its job. If you're summarizing a support ticket, you probably don't need the customer's full billing address in the prompt. This is the cheapest control you have and it's usually skipped because it takes an extra pass over the payload.

Redaction. For fields that are structurally identifiable — SSNs, card numbers, emails, phone numbers — run a redaction pass before the API call, not after. A regex-based pass catches the obvious formats; a named-entity-recognition pass catches names and addresses that don't follow a pattern. Neither is perfect. Say so honestly rather than claiming 100% coverage — legal has heard "should be fine" before and doesn't trust it.

What to bring: a description of what categories of PII flow through the feature, what you strip or mask before the API call, and an honest statement of what you don't catch. If you have no PII handling at all, that's fine to say too — as long as it's a deliberate statement ("this feature never sees customer PII because X") rather than an unexamined gap.

EU AI Act basics — keep this high-level and hedge appropriately

I'm not a lawyer and the AI Act's implementation timeline has moved before, so treat this section as a starting vocabulary, not a compliance determination — get real counsel for the actual classification.

The Act sorts AI systems into risk categories, and the obligations scale with the category:

• Unacceptable risk — banned outright (social scoring, certain biometric categorization). Most business features aren't here.
• High risk — systems used in employment decisions, credit scoring, biometric identification, and similar categories carry the heaviest obligations: risk management, documentation, human oversight, and conformity assessment.
• Limited risk — this is where most LLM-powered features land, including chatbots and generative content tools. The main obligation is transparency: users need to know they're interacting with an AI system, and AI-generated content in some contexts needs to be disclosed as such.
• Minimal risk — everything else, no specific obligations beyond general law.

What to bring: a plain statement of which category you believe your feature falls into and why, plus whatever transparency measures you've already built (a disclosure that the chat is AI-powered, a label on AI-generated summaries). Don't try to self-certify a high-risk classification in an engineering meeting — that's exactly the kind of determination legal needs to make, and your job is to hand them the facts about what the system does, not the legal conclusion.

Audit trail expectations

This is the question engineers are most likely to have skipped, because it doesn't show up until something goes wrong and someone asks "what did the model actually see and say on March 3rd."

Minimum bar, in my experience:

• Log which model version and provider handled each request — model behavior changes between versions, and "which model generated this output" is often the first question in an incident review.
• Log a reference to the input and output, even if the full content is redacted or expired per your retention policy — a hash or an ID that lets you correlate with other systems is better than nothing.
• Log the timestamp and, if the feature makes decisions (not just generates text), which downstream action was taken as a result.

You don't need a full replay system for a first version of this. You do need to be able to answer "what happened" for a specific request from a specific day, for whatever your retention window is.

The checklist to bring into the room

• Which API, which region, which subprocessors
• Vendor's stated retention window, cited from the actual DPA
• Confirmation you're on the API tier, not a consumer account
• Confirmation no-training is contractual, and whether you're in any opt-in improvement program
• What PII the feature touches, what you strip/redact before the call, and what you don't catch
• Your best-guess AI Act risk category and current transparency measures
• What you log per request and for how long

None of this is exotic. It's the difference between "we'll get back to you" and a signed-off feature in the same meeting. Bring answers, not vibes.